Lost property notification method, device, electronic device, storage medium, and computer program

Abstract

The present disclosure relates to a lost item notification method, device, electronic device, and storage medium, which includes detecting a lost item in a cabin of a vehicle, determining identity information of a target occupant who is the owner of the lost item in response to the detection of the lost item, and generating notification information based on the identity information.

Description

[Technical field]

[0001] The present disclosure relates to vehicles and computer vision technology, and in particular to a lost property notification method, device, electronic device, and storage medium. [Background technology]

[0002] When people use vehicles, they may forget their belongings (wallets, keys, etc.) on the vehicle, which may cause unnecessary trouble or loss. Therefore, detection and notification of lost items on vehicles is of great significance. Summary of the Invention

[0003] The present disclosure provides a method and apparatus for lost item notification.

[0004] According to one aspect of the present disclosure, Detecting lost property within a cabin of a vehicle; In response to the detection of the lost item, determining identity information of a target occupant who is an owner of the lost item; generating notification information based on the identity information; A method for notifying lost property is provided, including:

[0005] According to one aspect of the present disclosure, a first detection module for detecting lost items within a cabin of the vehicle; an identification module that, in response to the detection of the lost item, identifies an identity of a target occupant who is an owner of the lost item; A lost-item notification device is provided, comprising: a first generating module for generating notification information based on the identity information.

[0006] According to one aspect of the present disclosure, there is provided an electronic device including one or more processors and a storage memory storing executable instructions, the one or more processors configured to perform the above method by invoking the executable instructions stored in the storage memory.

[0007] According to one aspect of the present disclosure, there is provided a computer readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, causing implementation of the above method.

[0008] According to one aspect of the present disclosure, there is provided a computer program product comprising computer readable code which, when executed in an electronic device, causes a processor of the electronic device to execute instructions for implementing the above method.

[0009] In an embodiment of the present disclosure, lost items are detected within the cabin of a vehicle, and upon detection of a lost item, the identity information of a target occupant who is the owner of the lost item is identified, and notification information is generated based on the identity information. This makes it possible to provide a notification based on the identity information of the occupant who is the owner of the lost item within the cabin of the vehicle, and to provide a target-oriented notification, thereby improving the effectiveness of lost item notifications and enhancing the property security of occupants.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0011] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0012] The drawings, which are included as part of the specification, illustrate embodiments of the present disclosure and, together with the specification, explain the technical means of the present disclosure. [Figure 1] 1 shows a flowchart of a lost-item notification method according to an embodiment of the present disclosure. [Diagram 2] A schematic diagram of an application scenario for private vehicles is shown below. [Diagram 3] A schematic diagram of an application scenario for car sharing vehicles is shown. [Figure 4] A schematic diagram of an application scenario for online vehicle dispatch is shown below. [Diagram 5] 1 shows a block diagram of a lost-item notification device according to an embodiment of the present disclosure. [Figure 6] 8 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. [Figure 7] 19 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Various exemplary embodiments, features, and aspects of the present disclosure will now be described in detail with reference to the drawings, in which like reference numerals represent elements of the same or similar function. While illustrating various aspects of the embodiments in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0014] The word "exemplary" herein means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or superior over other embodiments.

[0015] In this specification, the term "and / or" merely describes a related relationship between related objects and indicates that three relationships can exist, for example, A and / or B can indicate three cases: only A exists, A and B exist simultaneously, and only B exists. In addition, in this specification, the term "at least one" indicates any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can indicate including any one or more elements selected from a set consisting of A, B, and C.

[0016] In addition, in order to better explain the present disclosure, many specific details are shown in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can be similarly implemented without any specific details. In some embodiments, in order to emphasize the gist of the present disclosure, detailed descriptions are not provided for methods, means, elements and circuits that are well known to those skilled in the art.

[0017] In an embodiment of the present disclosure, lost items are detected within the cabin of a vehicle, and upon detection of a lost item, the identity information of a target occupant who is the owner of the lost item is identified, and notification information is generated based on the identity information. This makes it possible to provide a notification based on the identity information of the occupant who is the owner of the lost item within the cabin of the vehicle, and to provide a target-oriented notification, thereby improving the effectiveness of lost item notifications and enhancing the property security of occupants.

[0018] FIG. 1 shows a flowchart of a lost-item notification method according to an embodiment of the present disclosure. The lost-item notification method may be executed by a lost-item notification device. In one possible embodiment, the lost-item notification method may be executed by a terminal device, a server, or other processing device. The terminal device may be an in-vehicle device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, or a wearable device, etc. The in-vehicle device may be an infotainment or domain controller in the cabin, or a host for performing data processing operations such as images in an OMS (Occupant Monitoring System) or a DMS (Driver Monitor System), etc. In some possible embodiments, the lost-item notification method may be realized by a processor invoking computer-readable instructions stored in a storage memory. As shown in FIG. 1, the lost-item notification method includes steps S11 to S13.

[0019] In step S11, a lost item is detected within the cabin of the vehicle.

[0020] In the embodiment of the present disclosure, the vehicle may be a car or a vehicle for carrying people or cargo, such as a ship or an aircraft. Accordingly, the cabin of the vehicle may be a passenger compartment, a cabin, an interior of an aircraft, etc. The vehicle may be a private car or a managed car, for example, a car-sharing car, an online car dispatching car, a taxi, a bus, a school bus, a long-distance or short-distance truck, a long-distance or short-distance passenger car, a train, a subway, a streetcar, etc.

[0021] In one possible embodiment, performing the above-mentioned detection of lost items within a cabin of a vehicle may include obtaining a video stream within the cabin of the vehicle, and performing the detection of lost items within the cabin based on the video stream within the cabin.

[0022] In this embodiment, a video stream of the cabin may be collected by a camera installed in the cabin. For example, the camera installed in the cabin may include an OMS camera, a DMS camera, etc. An OMS may perform other occupant detection functions, including detecting occupant attributes (age, gender, mood state, physical state, etc.), detecting occupant gestures, etc. As an example of this embodiment, a processor may obtain a video stream of the cabin of the vehicle from a camera installed in the cabin, and perform lost item detection in the cabin based on the video stream of the cabin.

[0023] In this embodiment, detecting lost items within the cabin may include at least one of detecting the presence or absence of lost items, detecting the type of lost items, detecting the location of lost items, and the like.

[0024] As an example of this embodiment, detecting lost items within the cabin based on a video stream within the cabin as described above may include detecting whether or not there is lost item within the cabin based on a video stream within the cabin.

[0025] In one possible embodiment, an image before all occupants enter the cabin may be obtained as a reference image, and any lost items in the cabin may be identified by comparing an image after all occupants have left the cabin with the reference image.

[0026] In another possible embodiment, for each occupant, an image before the occupant enters the cabin is obtained as a reference image corresponding to the occupant, and an image after the occupant leaves the cabin is compared with the reference image corresponding to the occupant to confirm items lost in the cabin of the vehicle during the occupant's period of use of the vehicle. The image before the occupant enters the cabin may include a video frame immediately before a video frame in which the occupant is first detected in the video stream of the cabin. The image after the occupant leaves the cabin may include a video frame immediately after a video frame in which the occupant is last detected in the video stream of the cabin.

[0027] In an embodiment of the present disclosure, detecting lost items within the cabin may include at least one of detecting the presence or absence of lost items, detecting the type of lost items, detecting the location of lost items, and the like.

[0028] In one possible embodiment, an image in which the occupant has left the cabin may be compared with a reference image to determine whether or not an item that is not in the reference image is present in the image in which the occupant has left the cabin. If such an item is present, it can be determined that there is a lost item. On the other hand, if such an item is not present, it can be determined that there is no lost item. In another possible embodiment, an image in which the occupant has left the cabin may be compared with a reference image to determine whether or not an item that corresponds to a predetermined type that is not in the reference image is present in the image in which the occupant has left the cabin. If such an item is present, it can be determined that there is a lost item. On the other hand, if such an item is not present, it can be determined that there is no lost item.

[0029] In one possible embodiment, in addition to detecting whether a lost item is present, the type of the lost item may be detected. In another possible embodiment, in response to detecting a lost item, a type may be identified for the detected lost item, thereby identifying the type of the lost item.

[0030] In one possible embodiment, the presence or absence of a lost item and / or the type of lost item may be detected, and the location of the lost item may be detected. In another possible embodiment, the location of the lost item may be detected in response to the lost item being detected.

[0031] In an embodiment of the present disclosure, a lost item detection result in the cabin of a vehicle can be obtained by performing lost item detection in the cabin. When the lost item detection result includes that a lost item has been detected, the lost item detection result may further include information of the lost item.

[0032] In one possible embodiment, the information about the lost item may include a type of the lost item, for example, a wallet, keys, a mobile phone, an umbrella, a briefcase, a backpack, a suitcase, or a bottle.

[0033] In another possible embodiment, the lost item information may include the location of the lost item within the cabin.

[0034] As an example of this embodiment, the location of the lost item in the cabin may include a general location of the lost item in the cabin. For example, the location of the lost item in the cabin may be a driver's seat area, a passenger seat area, a rear seat area, a trunk area, etc. Or, the location of the lost item in the cabin may be above the driver's seat, above the passenger seat, above the rear seat, in the trunk, below the driver's seat, below the passenger seat, below the rear seat, on the floor in front of the driver's seat, on the floor in front of the passenger seat, on the floor of the rear seat, etc.

[0035] As another example of this embodiment, the location of the lost item within the cabin may include a range or exact location of the lost item within the cabin. In one example, the location of the lost item within the cabin may include image coordinates of the lost item in an image coordinate system corresponding to an image of a video stream within the cabin. In another example, the location of the lost item within the cabin may include spatial coordinates of the lost item in a spatial coordinate system within the cabin.

[0036] In another possible embodiment, the information on the lost item may include an image of the lost item. As an example of this embodiment, the image of the lost item may be obtained by cutting out a region in which the lost item is located from one or more video frames of the video stream of the cabin that includes the lost item. As another example of this embodiment, the image of the lost item may be one or more video frames of the video stream of the cabin that includes the lost item. As another example of this embodiment, the lost item may be marked based on the position of the lost item in the video frame that includes the lost item, and the video frame in which the lost item is marked may be the image of the lost item. In this example, the mark of the lost item may be indicated in the video frame that includes the lost item, for example, by indicating the lost item with a box or an arrow that marks the lost item.

[0037] In step S12, in response to detection of a lost item, identification information of a target occupant who is the owner of the lost item is identified.

[0038] In embodiments of the present disclosure, the vehicle occupants may include any person riding in the vehicle. For example, the vehicle occupants may include the driver and / or non-drivers of the vehicle. For example, in a personal vehicle application, the non-drivers may be family members or friends of the vehicle owner, etc. In a ride-hailing application, the non-drivers may be passengers, etc.

[0039] In an embodiment of the present disclosure, the target occupant who is the owner of the lost item is the occupant who lost the lost item in the cabin. The identity information of the target occupant may be information related to the identity of the target occupant. For example, the identity information of the target occupant may include at least one of the target occupant's name, ID number, account, code, phone number, face image, etc.

[0040] In an embodiment of the present disclosure, in response to detection of a lost item, a target occupant who is the owner of the lost item may be identified based on an association relationship between the position of the lost item in the cabin and the position of the occupant in the cabin, and identity information of the target occupant may be acquired by performing face recognition on the target occupant. That is, identity information of the target occupant who is the owner of the lost item may be identified based on an association relationship between the position of the lost item and the occupant in the cabin. Alternatively, in an embodiment of the present disclosure, in response to detection of a lost item, identity information of the target occupant who is the owner of the lost item may be identified based on a time when the lost item appeared in the cabin and a time when the occupant used the vehicle. That is, identity information of the target occupant who is the owner of the lost item may be identified based on an association relationship between the time when the lost item appeared in the cabin and a time when the occupant used the vehicle. Alternatively, as a selectable form, in response to detection of a lost item, attributes related to a user group, such as the color and type of the lost item, may be identified, and then identity information of the target occupant who is the owner of the lost item may be identified based on a degree of association between the person who appeared in the cabin and the attribute of the lost item. For example, if the lost item is a woman's shoulder bag and it is detected that only one female occupant was present in the cabin during the corresponding time period, the lost woman's shoulder bag can be identified as the lost item of this woman.

[0041] In step S13, notification information is generated based on the identity information.

[0042] In one possible embodiment, the identity information may be included in the notification information. For example, if the identity information includes the name "Zhang San", notification information such as "Mr. Zhang, your mobile phone seems to have been left in the vehicle, please return it as soon as possible" may be generated. In another possible embodiment, notification information may be generated based on the identity information of the target occupant for sending to the terminal device of the target occupant. For example, notification information may be generated that "there is a lost item in the vehicle", and the information location setting is "send to mobile phone 133********".

[0043] In another possible embodiment, the form of the corresponding notification information may differ depending on the identity information. For example, the form of the notification information may be text notification information, audio notification information, image notification information, video notification information, etc. As an example of this embodiment, notification information may be generated in the form of notification information corresponding to the identity information. For example, when the form of notification information corresponding to the identity information is audio notification information, audio notification information may be generated. As another example of this embodiment, image notification information may be generated for cutting out an image inside the cabin including a lost item and transmitting it to the mobile phone of the target occupant. As another example of this embodiment, notification information may be generated in the form of notification information corresponding to the identity information, to notify the target occupant corresponding to the identity information. For example, when the form of notification information corresponding to the identity information is video notification information and the lost item is a mobile phone, video notification information may be generated based on the identity information of the target occupant that a target person in the video corresponding to the identity information of the target occupant has left his mobile phone in the vehicle. For example, when the target occupant is a middle-aged man, the target person in the video corresponding to the identity information of the target occupant may be a middle-aged man.

[0044] In an embodiment of the present disclosure, lost items are detected within the cabin of a vehicle, and upon detection of lost items, identity information of a target occupant who is the owner of the lost item is identified, and notification information is generated based on the identity information. This makes it possible to provide notification of lost items based on the identity information of the occupant who is the owner of the lost item within the cabin of the vehicle, and to provide notification of lost items in a target-oriented manner, thereby improving the effectiveness of lost item notifications and enhancing the property security of occupants.

[0045] An application scenario of the embodiment of the present disclosure is described as follows. In this application scenario, a camera installed in a cabin of a vehicle may collect a video stream in the cabin. The camera may send the video stream in the cabin to a processor. The processor may detect a lost item in the cabin based on the video stream in the cabin. The processor may identify the identity information of a target occupant who is the owner of the lost item according to the detection of the lost item. For example, if the identity information includes the name "Zhang San" and the lost item is a mobile phone, the processor may generate a text notification information saying "Mr. Zhang, your mobile phone seems to have been left in the vehicle. Please return it as soon as possible" and send the text notification information to the mobile phone of the target occupant. At the same time, the light of the vehicle may be controlled to flash, so that the target occupant who has just left the vehicle returns to the vehicle to retrieve the lost item.

[0046] In one possible embodiment, determining the identity of a target occupant who is the owner of the lost item in response to the detection of the lost item includes determining a location of the lost item within the cabin in response to the detection of the lost item, and determining the identity of a target occupant who is the owner of the lost item based on the location of the lost item within the cabin.

[0047] As an example of this embodiment, the detection of a lost item in the cabin of a vehicle may include detecting a position of the lost item in the cabin, and the information of the lost item may include a position of the lost item in the cabin. According to this example, the position of the lost item in the cabin of a vehicle can be obtained by detecting a lost item in the cabin of a vehicle. As another example of this embodiment, if the position of the lost item in the cabin has not been detected in step S11, the detection of the position of the lost item in the cabin may be continued after step S11 based on a video stream in the cabin in response to the detection of the lost item.

[0048] As one example of this embodiment, the location of the lost item within the cabin may include a location where the lost item first appeared within the cabin. As another example of this embodiment, the location of the lost item within the cabin may include a location where the lost item last appeared within the cabin. As another example of this embodiment, the location of the lost item within the cabin may include multiple locations where the lost item appeared within the cabin. For example, the location of the lost item within the cabin may include each location where the lost item appeared within the cabin.

[0049] In this embodiment, a target occupant who is the owner of the lost item may be identified based on the correlation between the location of the lost item in the cabin and the location of the occupant in the cabin, and facial recognition may be performed on the target occupant to identify the identity information of the target occupant.

[0050] In this embodiment, in response to detection of a lost item, the location of the lost item in the cabin is identified, and the identity information of the target occupant who is the owner of the lost item is identified based on the location of the lost item in the cabin. Since the location of the lost item and the location of the target occupant who is the owner of the lost item are usually closely related, according to this embodiment, the relationship between the location of the lost item and the location of the occupant can increase the accuracy of the identity information of the target occupant who is the owner of the lost item in the cabin.

[0051] As an example of this embodiment, the step of identifying the identity information of the target occupant who is the owner of the lost item based on the position of the lost item in the cabin includes identifying a related area of ​​the lost item in the cabin based on the position of the lost item in the cabin, performing face recognition on the related area based on a video stream in the cabin to obtain a face recognition result of the related area, and identifying the identity information of the target occupant who is the owner of the lost item based on the face recognition result of the related area. The related area may include a spatial area within a certain distance around the lost item, for example, a three-dimensional spatial coordinate area extending outward from the position coordinates of the lost item as a center. Alternatively, the related area may be a predetermined spatial area of ​​the vehicle cabin that includes the position of the lost item. Here, the predetermined spatial area of ​​the vehicle cabin may be obtained by dividing an area in the vehicle cabin, and may include, for example, a driver area, a passenger area, a left rear area, a right rear area, etc.

[0052] In one example, the location of the lost item within the cabin may include image coordinates of the lost item in an image coordinate system to which an image of the video stream within the cabin corresponds, and the relevant region may include a relevant image coordinate region of the image coordinate system in which the lost item is located, and performing facial recognition on the video stream within the cabin based on the relevant region may include performing facial recognition on the relevant region of the video stream within the cabin.

[0053] In another example, the location of the lost item within the cabin may include spatial coordinates of the lost item in a spatial coordinate system within the cabin. The region of interest may include a related spatial coordinate region of the lost item in a spatial coordinate region within the cabin. Performing facial recognition on the video stream within the cabin based on the region of interest includes determining an image coordinate region to which the related spatial coordinate region corresponds, and performing facial recognition on the image coordinate region of the video stream within the cabin to which the related spatial coordinate region corresponds.

[0054] In one example, the related area of ​​the lost item in the cabin may be a predetermined area determined by using the coordinates of the lost item in the cabin as a geometric center. For example, the related area of ​​the lost item in the cabin may be a coordinate area of ​​a related image of a predetermined size determined by using the image coordinates of the lost item in an image coordinate system to which an image of a video stream in the cabin corresponds as a geometric center. Or, the related area of ​​the lost item in the cabin may be a coordinate area of ​​a related space of a predetermined size determined by using the spatial coordinates of the lost item in a spatial coordinate system in the cabin as a geometric center.

[0055] In another example, a correspondence between the position of an item in the cabin and a related area of ​​the item in the cabin may be created in advance. This allows the related area of ​​the lost item in the cabin to be identified based on the previously created correspondence and the position of the lost item in the cabin. For example, if the position of the lost item in the cabin is within a first predetermined coordinate area, the related area of ​​the lost item in the cabin may be a driver's seat area. If the position of the lost item in the cabin is within a second predetermined coordinate area, the related area of ​​the lost item in the cabin may be a passenger seat area. If the position of the lost item in the cabin is within a third predetermined coordinate area, the related area of ​​the lost item in the cabin may be a rear seat area.

[0056] In one example, face recognition may be performed on a passenger in the relevant area who has at least a part of his / her body in contact with the lost property, and a face recognition result may be obtained. In another example, face recognition may be performed on all passengers in the relevant area, and a face recognition result may be obtained.

[0057] In this example, a related area of ​​the lost item in the cabin is identified based on the location of the lost item in the cabin, face recognition is performed on the related area based on the video stream in the cabin to obtain a face recognition result for the related area, and identity information of a target occupant who is the owner of the lost item is identified based on the face recognition result for the related area. In this way, an image of an area that is positionally associated with the lost item can be extracted from a video frame included in the video stream, and the accuracy of the identified identity information of the target occupant can be increased by face recognition technology, while the computational complexity of identifying the target occupant can be reduced.

[0058] In one example, performing facial recognition on the relevant area based on the video stream within the cabin and obtaining a facial recognition result for the relevant area includes determining a target video frame from the video stream based on a time when the lost item first appeared in the video stream within the cabin, and performing facial recognition on a relevant area of ​​the target video frame and obtaining a facial recognition result for the relevant area.

[0059] In this example, the time when the lost item first appears in the video stream in the cabin may be a time when a first video frame including the lost item is located in the video stream in the cabin. The number of the target video frames may be one or more. For example, the target video frame may include a first video frame including the lost item in the video stream in the cabin. Or, the target video frame may further include a plurality of video frames after the first video frame including the lost item. Or, the target video frame may further include a plurality of video frames before the first video frame including the lost item.

[0060] In this example, a target video frame is determined from the video stream according to a time when the lost item first appears in the video stream in the cabin, face recognition is performed on a relevant region of the target video frame, face recognition results are obtained for the relevant region, and the video frames related to the lost item are further extracted from the time dimension so as to reduce the number of video frames that need to be analyzed, which is favorable for improving the efficiency of identifying the identity information of the target occupant and also favorable for improving the accuracy of the identified identity information of the target occupant.

[0061] In one example, identifying identity information of a target occupant who is the owner of the lost item based on the facial recognition results of the related area as described above includes determining candidate occupants for the lost item based on the facial recognition results of the related area, and identifying identity information of a target occupant who is the owner of the lost item from among the candidate occupants who entered the cabin earlier or at the same time as the lost item first appeared in the video stream and left the cabin later than the lost item first appeared in the video stream.

[0062] In this example, the face recognition result of the related area may include the identity information of each detected face. The occupants to which the identity information of each face included in the face recognition result corresponds may be the candidate occupants. Alternatively, among the occupants to which the identity information of each face included in the face recognition result corresponds, an occupant whose body at least part comes into contact with the lost property may be the candidate occupant. The candidate occupants may be one or more. When there is one candidate occupant, the identity information of the candidate occupant may be identified as the identity information of a target occupant who is the owner of the lost property, or it may be further determined whether the candidate occupant is the target occupant based on a result of a match between the candidate occupant and the appearance time of the lost property. When there are multiple candidate occupants, the identity information of the target occupant who is the owner of the lost property may be identified by screening multiple candidate occupants based on the time when the candidate occupant entered the cabin and the time when he / she left the cabin. In this example, the time when the occupant entered the cabin and the time when he / she left the cabin may be determined by detecting the occupant from a video stream in the cabin.

[0063] In this example, candidate occupants for the lost item are determined based on the facial recognition results of the relevant area, and the identity information of the target occupant who is the owner of the lost item is identified from among the candidate occupants who entered the cabin earlier or at the same time as the lost item first appeared in the video stream and left the cabin later than the lost item first appeared in the video stream. This can further increase the consistency between the times when the lost item and the target occupant appeared in the video stream within the cabin, thereby improving the accuracy of the identity information of the identified target occupant.

[0064] In one example, among the candidate occupants whose time of entry into the cabin is earlier or the same as the time of the lost item first appearing in the video stream and whose time of leaving the cabin is later than the time of the lost item first appearing in the video stream, if there are multiple occupants whose body parts touch the lost item, the identity information of the candidate occupant who touched the lost item earliest may be specified as the identity information of the target occupant who is the owner of the lost item. In another example, among the candidate occupants whose time of entry into the cabin is earlier or the same as the time of the lost item first appearing in the video stream and whose time of leaving the cabin is later than the time of the lost item first appearing in the video stream, if there are multiple occupants whose body parts touch the lost item, the identity information of the candidate occupant who touches the lost item the longest may be specified as the identity information of the target occupant who is the owner of the lost item. Alternatively, among the candidate occupants who entered the cabin at a time earlier than or the same as the time when the lost item first appeared in the video stream and left the cabin at a time later than the time when the lost item first appeared in the video stream, if there are a plurality of occupants whose body parts touched the lost item, the target occupant who is the owner of the lost item can be further determined based on other attribute information of the lost item related to the characteristics of the occupant who is the owner of the lost item, such as the gender, age, etc. For example, if the detected lost item is a shoulder bag for a woman, the probability that the lost item is the lost item of a female occupant out of a male occupant and a female occupant whose body parts touched the lost item is higher than the probability that the lost item is the lost item of a male occupant.

[0065] In another example of this embodiment, a target occupant who is the owner of the lost item may be identified based on the location of the lost item in the cabin and the location of at least one part of the occupant's body in the cabin, and identity information of the target occupant may be acquired by performing face recognition on the target occupant. For example, if occupant A is holding lost item B when he / she enters the cabin (e.g., occupant A was holding lost item B when he / she got in the cabin), it can be determined that the target occupant who is the owner of lost item B is occupant A.

[0066] In one possible embodiment, determining the identity of a target occupant who is the owner of the lost item in response to a lost item being detected includes determining the identity of a target occupant who is the owner of the lost item in response to a lost item being detected based on a time when the lost item first appears in a video stream within the cabin.

[0067] In this embodiment, for application scenarios in which the number of passengers simultaneously appearing in the cabin is small, such as private cars, Internet car dispatching, car sharing cars, etc., only the time when the lost item first appears in the video stream in the cabin may be considered, and the identity information of the target passenger who is the owner of the lost item may be identified based on the time when the lost item first appears in the video stream in the cabin. According to this embodiment, the identity information of the target passenger who is the owner of the lost item can be quickly and conveniently identified.

[0068] Of course, in other embodiments, for application scenarios such as private cars, internet car dispatch, and car sharing cars, the time when the lost item first appears in the video stream in the cabin may be considered, as well as the location of the lost item in the cabin. For example, when there are multiple occupants in a private car, the time when the lost item first appears in the video stream in the cabin and the location of the lost item in the cabin can be considered simultaneously to accurately identify the target occupant who is the owner of the lost item. Or, in a car sharing scenario for internet car dispatch, the time when the lost item first appears in the video stream in the cabin and the location of the lost item in the cabin can be considered simultaneously to accurately identify the target occupant (car sharing user or driver) who is the owner of the lost item.

[0069] As an example of this embodiment, identifying the identity information of a target occupant who is the owner of the lost item based on the time when the lost item first appeared in the video stream inside the cabin includes obtaining a usage history of the vehicle including user identity information and corresponding usage times, identifying the identity information of a target user based on the usage history, whose corresponding usage time matches the time when the lost item first appeared in the video stream, and identifying the identity information of the target user as the identity information of the target occupant who is the owner of the lost item.

[0070] For example, in the application of the online car dispatch, the vehicle may be an online car dispatch, the use history of the vehicle may be the order history of the online car dispatch, the user may be a passenger, and the corresponding use time may be the time the passenger rode in the online car dispatch. The order history of the online car dispatch is obtained, and based on the order history of the online car dispatch, the identity information of the target passenger whose time on the online car dispatch coincides with the time when the lost item first appeared in the video stream is identified, and the identity information of the target passenger is identified as the identity information of the target passenger who is the owner of the lost item. In this way, the identity information of the passenger who is the owner of the lost item can be accurately identified in the online car dispatch. Of course, in the application of the online car dispatch, the target passenger may be the driver of the online car dispatch.

[0071] Alternatively, in the application of a car-sharing vehicle, the vehicle may be a car-sharing vehicle, the use history of the vehicle may be a rental history of the car-sharing vehicle, the user may be a renter, and the corresponding use time may be the time when the renter rented the car-sharing vehicle. The rental history of the car-sharing vehicle is acquired, and based on the rental history of the car-sharing vehicle, the identity information of a target renter whose rental time of the car-sharing vehicle matches the time when the lost item first appeared in the video stream is identified, and the identity information of the target renter is identified as the identity information of a target passenger who is the owner of the lost item. In this way, the identity information of the renter who lost the lost item in the car-sharing vehicle can be accurately identified. Of course, in the application of a car-sharing vehicle, the target passenger may be a staff member of the operating company of the car-sharing vehicle, etc.

[0072] Or, in the application scenario of a private car, the owner of the private car may allow a family member or friend to drive the car. By acquiring the usage history of the private car, the identity information of the target user whose usage time of the private car coincides with the time when the lost item first appears in the video stream can be identified, so that the identity information of the family member or friend who lost the lost item in the private car can be accurately identified.

[0073] In this example, the usage history of the vehicle is acquired, and based on the usage history, the identity information of the target user whose corresponding usage time coincides with the time when the lost item first appeared in the video stream is identified, and the identity information of the target user is identified as the identity information of the target passenger who is the owner of the lost item. By effectively using the usage history of the vehicle, the identity information of the target passenger can be identified more quickly and accurately, so that when the vehicle is registered and used, notification information for the target passenger who is the owner of the lost item can be generated quickly and accurately.

[0074] In one possible embodiment, the lost item notification method further includes, before detecting a lost item in the cabin of the vehicle as described above, acquiring images and / or videos of an occupant entering the cabin, detecting information of an item carried by the occupant based on the images and / or videos of the occupant entering the cabin, and associating the information of the item carried by the occupant with identification information of the occupant, and, in response to the detection of a lost item, identifying the identification information of a target occupant who is the owner of the lost item includes identifying the identification information of the occupant associated with the lost item as the identification information of the target occupant who is the owner of the lost item in response to the detection of a lost item.

[0075] In this embodiment, a camera mounted within the cabin may capture images and / or video of occupants as they enter the cabin, and a camera mounted outside the cabin of the vehicle may capture images and / or video of occupants as they enter the cabin.

[0076] In this embodiment, the information of the item carried by the occupant may include at least one of an image of the item carried by the occupant, feature information extracted from the image of the item carried by the occupant, the type of item carried by the occupant, and attributes of the item carried by the occupant (e.g., one or more of color, size, pattern), etc.

[0077] In this embodiment, the occupant information may include at least one of identification information of the occupant and / or facial information of the occupant, etc. The occupant facial information may include a facial image and / or facial features of the occupant.

[0078] In this embodiment, images and / or videos of an occupant entering the cabin are obtained, and information of an item carried by the occupant is detected based on the images and / or videos of the occupant entering the cabin. The information of the item carried by the occupant is associated with the occupant's identification information. Upon detection of a lost item, the identification information of the occupant associated with the lost item is identified as the identification information of a target occupant who is the owner of the lost item. This makes it possible to quickly and accurately identify the identification information of the target occupant who is the owner of the lost item when a loss occurs.

[0079] In one possible embodiment, the generating of notification information based on the identity information as described above includes generating first notification information based on the identity information, and the lost item notification method further includes, after generating notification information based on the identity information as described above, sending the first notification information to a target terminal corresponding to the identity information.

[0080] In this embodiment, the target terminal corresponding to the identity may be the target passenger's terminal. For example, the target terminal may be the target passenger's mobile phone, tablet, wearable smart device, etc. In this embodiment, the first notification information is sent to the target terminal corresponding to the identity, so that the target passenger can be immediately notified that the lost item has been lost in the vehicle.

[0081] As an example of this embodiment, the first notification information may be generated based on information about the lost item. For example, the first notification information may be generated based on at least one of the type, image, and location of the lost item. This allows the target passenger to be informed of intuitive information about the lost item, which can help the target passenger quickly decide whether to return to the vehicle and retrieve the lost item.

[0082] In one possible embodiment, generating notification information based on the identification information includes generating second notification information based on the identification information and the lost item information, and the lost item notification method further includes, after generating the notification information based on the identification information, sending the second notification information to a management terminal connected to the vehicle and / or a management server connected to the vehicle.

[0083] In this embodiment, the information on the lost item may include at least one of the type, image, location, etc. of the lost item. In this embodiment, the management terminal connected to the vehicle may be a terminal associated with the vehicle's operating company (e.g., a terminal of a staff member of the operating company), etc., and the management server connected to the vehicle may be a server of a platform operated by the vehicle's operating company, etc.

[0084] In this embodiment, by sending the second notification information to a management terminal connected to the vehicle and / or a management server connected to the vehicle, if the target occupant does not promptly retrieve the lost item, the manager (e.g., the operating company) can promptly notify the target occupant or promptly take custody of the target occupant's lost item.

[0085] As one example of this embodiment, generating second notification information based on the identification information and the lost property information as described above includes generating second notification information based on at least one of the vehicle ID information, the boarding location of the target occupant and the disembarking location of the target occupant, the identification information and the lost property information.

[0086] In this embodiment, the vehicle ID information may be information that can uniquely identify the vehicle, for example, the vehicle ID information may be one of the following: a license plate number of the vehicle, a code of the vehicle, an identification number of the vehicle driver, a telephone number of the vehicle driver, etc.

[0087] In this embodiment, a video frame in which the target occupant first appears and a video frame in which the target occupant last appears may be determined from the video stream in the cabin, a boarding time of the target occupant may be determined based on the time of the video frame in which the target occupant first appears, and a disembarking time of the target occupant may be determined based on the time of the video frame in which the target occupant last appears. A boarding location of the target occupant may be determined based on geolocation information of the position of the vehicle at the boarding time. A disembarking location of the target occupant may be determined based on geolocation information of the position of the vehicle at the disembarking time.

[0088] In this embodiment, by generating second notification information based on the vehicle ID information, at least one of the target occupant's boarding location and the target occupant's disembarking location, the identity information and the lost property information, it is advantageous for the vehicle manager (e.g., an operating company) to record more relevant information of the lost property, thereby contributing to finding the target occupant and managing the lost property.

[0089] In one possible embodiment, the generating of notification information based on the identification information includes generating third notification information based on the identification information and the lost item information if the target occupant is not the driver of the vehicle, and the lost item notification method further includes sending the third notification information to a terminal of the driver of the vehicle; and / or the generating of notification information based on the identification information includes generating fourth notification information based on the identification information, and the lost item notification method further includes generating control information for controlling an in-vehicle device installed in the cabin to output the fourth notification information.

[0090] In one example of this embodiment, the in-vehicle device may include a speaker, and the fourth notification information may include a voice for notifying the target occupant to retrieve the lost item, and may further include a notification sound. For example, the voice for notifying the target occupant to retrieve the lost item may be something like "Dear XX, your mobile phone seems to have been left in the vehicle. Please retrieve it as soon as possible."

[0091] In one embodiment, the in-vehicle device may include a display screen, which may output the fourth notification information in a manner such as flashing, playing a video, playing a text, etc. after receiving the control information.

[0092] In this embodiment, if the target occupant is not the driver of the vehicle, third notification information is generated based on the identity information and the information on the lost property, and the third notification information is sent to the terminal of the driver of the vehicle. This can notify the driver of the vehicle to keep the lost property of the target occupant for a while. Fourth notification information is generated based on the identity information, and control information is generated to control the in-vehicle device installed in the cabin to output the fourth notification information. Based on this, the occupant in the cabin can be notified, and the occupant in the cabin can take appropriate safety storage measures as soon as possible to reduce the risk of losing items.

[0093] In one possible embodiment, the method for notifying of lost property further comprises generating control information for controlling the blinking of indicator lights of the vehicle in response to the target occupant leaving the cabin, such that the indicator lights blink in response to the control information. According to this embodiment, the target occupant who has just left the cabin can be notified to return to the cabin to retrieve the item.

[0094] In one possible embodiment, the method for notifying lost property further includes generating repeat notification information in response to receiving a repeat notification request returned from the target terminal after sending the first notification information to the target terminal corresponding to the identity. In this embodiment, by generating the repeat notification information in response to receiving a repeat notification request returned from the target terminal, if the target occupant cannot return to the cabin promptly within a short time to retrieve the lost property, the property safety of the occupant can be further protected by re-notifying.

[0095] As an example of this embodiment, the method of notifying lost items further includes at least one of the following: after generating the repeat notification information, transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of transmission of the first notification information reaching a first predetermined time; transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of reception of the repeat notification request reaching a second predetermined time; transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of transmission of the first notification information reaching a first predetermined time and it is detected that the lost item has not been taken out of the cabin; and transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of reception of the repeat notification request reaching a second predetermined time and it is detected that the lost item has not been taken out of the cabin.

[0096] In this example, the first and second predetermined times may be default times or times set by the target occupant. For example, the first and second predetermined times may be 30 minutes, 1 hour, etc. In this example, repeated notifications may be canceled in response to detecting that the lost item has been removed from the cabin. This can improve notification efficiency and reduce unnecessary repeated notifications.

[0097] As an example of this embodiment, the method for notifying the lost property further includes, after generating the repeat notification information, generating control information for controlling an in-vehicle device installed in the cabin to output the repeat notification information in response to the target occupant returning the lost property to the cabin. In this example, by generating control information for controlling an in-vehicle device installed in the cabin to output the repeat notification information in response to the target occupant returning the lost property to the cabin, the target occupant can be notified immediately after the target occupant returns the lost property to the cabin.

[0098] In one example, generating control information for controlling an in-vehicle device installed in the cabin to output the repeated notification information in response to the target occupant returning the lost item into the cabin may include generating control information for controlling an in-vehicle device installed in the cabin to output the repeated notification information based on the type and / or location of the lost item in response to the target occupant returning the lost item into the cabin. For example, if the type of lost item is a wallet and the location of the lost item is on the front floor of the rear seats, the in-vehicle device may transmit voice information saying "There is a lost wallet on the front floor of the rear seats."

[0099] In one possible embodiment, the method for notifying lost items further includes receiving a first notification setting request through the infotainment of the vehicle, and setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification based on the first notification setting request. For example, a user can set a dish and a bottle as item types that do not require notification, a wallet, a mobile phone, and a briefcase as item types that require notification, and a mobile phone as item types that require repeated notification through the infotainment. According to this embodiment, a user can flexibly set the lost item types that do not require notification, require notification, and require repeated notification through the infotainment according to his or her needs.

[0100] In one example of this embodiment, after setting at least one of the item types that do not require notification, the item types that require notification, and the item types that require repeated notification based on the first notification setting request, the user can change the setting again through the infotainment or the target terminal according to his / her needs. For example, after setting a bottle as an item type that does not require notification through the infotainment, if the user wants to change the setting, he / she can re-set the bottle as an item type that requires notification through the infotainment or the mobile phone.

[0101] In one possible embodiment, the lost item notification method further includes receiving a second notification setting request returned from the target terminal, and setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification based on the second notification setting request. According to this embodiment, a user can flexibly set the lost item type that does not require notification, requires notification, or requires repeated notification via the target terminal according to his / her needs.

[0102] In one example of this embodiment, after setting at least one of the item types that do not require notification, the item types that require notification, and the item types that require repeated notification based on the second notification setting request, the user can change the setting again through the infotainment or the target terminal according to his / her needs. For example, after the user sets an umbrella as an item type that requires notification through the mobile phone, if the user wants to change the setting, he / she can re-set the umbrella as an item type that does not require notification through the infotainment or the mobile phone.

[0103] FIG. 2 shows a schematic diagram of an application scenario of a private vehicle. In the example shown in FIG. 2, a camera installed in the cabin of the private vehicle may be used to collect a video stream in the cabin. The camera may transmit the video stream in the cabin to a processor. The processor may perform lost item detection in the cabin based on the video stream in the cabin. The processor may identify identity information of a target occupant who is the owner of the lost item in response to the detection of the lost item. For example, if the target occupant is a driver, the processor may generate first notification information based on the identity information and transmit the first notification information to the driver's mobile phone. For example, after receiving the first notification information by the mobile phone, the driver of the private vehicle can return the vehicle to retrieve the lost item (e.g., a wallet). The processor may further generate fourth notification information based on the identity information and generate control information for controlling an in-vehicle device installed in the cabin to output the fourth notification information. The driver's mobile phone may repeatedly feed back notification requests to the processor. The processor may generate repeat notification information in response to receiving the repeat notification request returned from the target terminal, and may further transmit the repeat notification information to the driver's mobile phone in response to detecting that the time interval between the current time and the transmission time of the first notification information reaches a first predetermined time and that the lost item has not been taken out of the cabin. The driver's mobile phone may further feed back a second notification setting request to the processor. The processor may set at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeat notification based on the second notification setting request. For example, if the first notification information notifies that the types of lost items include a tissue box and a bottle, the driver may feed back a second notification setting request by his mobile phone after receiving the first notification information, and set the tissue box and the bottle as item types that do not require notification.

[0104] FIG. 3 shows a schematic diagram of an application scenario of a car-sharing vehicle. In the example shown in FIG. 3, a camera installed in the cabin of the car-sharing vehicle may be used to collect a video stream in the cabin. The camera may transmit the video stream in the cabin to a processor. The processor may detect lost items in the cabin based on the video stream in the cabin. The processor may identify identity information of a target passenger who is the owner of the lost item in response to the detection of the lost item. For example, if the target passenger is a driver, the processor may generate first notification information based on the identity information and transmit the first notification information to the driver's mobile phone. The processor may further generate fourth notification information based on the identity information and generate control information for controlling an in-vehicle device installed in the cabin to output the fourth notification information. The processor may further generate second notification information based on the identity information and the information on the lost item and transmit the second notification information to a server of a platform operated by an operating company connected to the car-sharing vehicle. Based on the second notification information, staff of the car sharing vehicle operating company can store the lost item for the target occupant who cannot retrieve the lost item for a while, so that the target occupant can retrieve the lost item at a convenient time.

[0105] FIG. 4 shows a schematic diagram of an application scenario of the online car dispatch. In the example shown in FIG. 4, a camera installed in the cabin of the online car dispatch may be used to collect a video stream in the cabin. The camera may transmit the video stream in the cabin to a processor. The processor may detect lost items in the cabin based on the video stream in the cabin. The processor may identify the identity information of the target passenger who is the owner of the lost item in response to the detection of the lost item. If the target passenger is a passenger, the processor may generate first notification information based on the identity information and send the first notification information to the passenger's mobile phone. The processor may further generate third notification information based on the identity information and the information on the lost item, and send the third notification information to the mobile phone of the driver of the online car dispatch to notify the driver of the online car dispatch to keep the passenger's lost item. The processor may further generate fourth notification information based on the identity information, and generate control information for controlling an in-vehicle device installed in the cabin to output the fourth notification information. The processor may further generate second notification information based on the identity information and the lost property information, and transmit the second notification information to a server of a platform operated by an operating company connected to the online ride-hailing service.

[0106] In addition to private cars, cars for car sharing, and internet car dispatch, the embodiments of the present disclosure can be applied to other vehicles, such as publicly operated vehicles (including but not limited to railroad public transportation vehicles), ships, etc., but the present invention is not limited thereto.

[0107] It should be understood that the above method embodiments mentioned in this disclosure can be combined with each other to form an embodiment without violating the principles and logic. Due to the limited number of pages, detailed descriptions are omitted in this disclosure. In the above method of a specific embodiment, it can be understood by those skilled in the art that the specific execution order of each step should be determined according to its function and possible internal logic.

[0108] In addition, the present disclosure further provides a lost-item notification device, an electronic device, a computer-readable storage medium, and a program, any of which can be used to realize any one of the lost-item notification methods of the present disclosure, and the corresponding technical means and descriptions can be referred to the corresponding descriptions of the methods, and detailed descriptions are omitted.

[0109] 5 shows a block diagram of a lost-item notification device according to an embodiment of the present disclosure. As shown in FIG. 5, the lost-item notification device includes a first detection module 51 for detecting a lost item in a cabin of a vehicle, an identification module 52 for identifying the identity information of a target passenger who is the owner of the lost item in response to the detection of the lost item, and a first generation module 53 for generating notification information based on the identity information.

[0110] In one possible embodiment, the identification module 52, in response to detecting a lost item, identifies the location of the lost item within the cabin, and identifies the identity of the target occupant who is the owner of the lost item based on the location of the lost item within the cabin.

[0111] In one possible embodiment, the identification module 52 identifies a relevant area of ​​the lost item in the cabin based on the location of the lost item in the cabin, performs facial recognition on the relevant area based on a video stream in the cabin, obtains facial recognition results for the relevant area, and identifies identity information of a target occupant who is the owner of the lost item based on the facial recognition results for the relevant area.

[0112] In one possible embodiment, the identification module 52 determines a target video frame from the video stream based on the time when the lost item first appears in the video stream inside the cabin, performs facial recognition on a relevant area of ​​the target video frame, and obtains a facial recognition result for the relevant area.

[0113] In one possible embodiment, the identification module 52 determines candidate occupants for the lost item based on the facial recognition results of the relevant area, and identifies the identity information of a target occupant who is the owner of the lost item from among the candidate occupants whose time of entry into the cabin is earlier or the same as the time the lost item first appeared in the video stream and whose time of leaving the cabin is later than the time the lost item first appeared in the video stream.

[0114] In one possible embodiment, in response to detecting a lost item, the identification module 52 identifies the identity of the target occupant who is the owner of the lost item based on the time the lost item first appeared in the in-cabin video stream.

[0115] In one possible embodiment, the identification module 52 obtains a usage history of the vehicle including user identity information and corresponding usage times, and based on the usage history, identifies a target user identity information whose corresponding usage time matches the time when the lost item first appeared in the video stream, and identifies the target user identity information as the identity information of a target occupant who is the owner of the lost item.

[0116] In one possible embodiment, the lost item notification device further includes an acquisition module for acquiring images and / or videos of an occupant when he enters the cabin, a second detection module for detecting information of an item carried by the occupant based on images and / or videos of the occupant when he enters the cabin, and an association module for associating the information of the item carried by the occupant with the occupant's identification information, and in response to the detection of a lost item, the identification module 52 identifies the identification information of the occupant associated with the lost item as the identification information of a target occupant who is the owner of the lost item.

[0117] In one possible embodiment, the first generation module 53 generates first notification information based on the identity information, and the lost item notification device further includes a first transmission module for transmitting the first notification information to a target terminal corresponding to the identity information.

[0118] In one possible embodiment, the first generation module 53 generates second notification information based on the identity information and the lost item information, and the lost item notification device further includes a second transmission module for transmitting the second notification information to a management terminal connected to the vehicle and / or a management server connected to the vehicle.

[0119] In one possible embodiment, the first generation module 53 generates second notification information based on the vehicle ID information, at least one of the target passenger's boarding location and the target passenger's disembarking location, the identity information, and the lost property information.

[0120] In one possible embodiment, the first generation module 53 generates third notification information based on the identity information and the lost item information if the target occupant is not the driver of the vehicle, and the lost item notification device further includes a third transmission module for transmitting the third notification information to the vehicle driver's terminal, and / or the first generation module 53 generates fourth notification information based on the identity information, and the lost item notification device further includes a first control module for generating control information that controls an in-vehicle device installed in the cabin to output the fourth notification information.

[0121] In one possible embodiment, the lost-item notification device further comprises a second generating module for generating repeated notification information in response to receiving a repeated notification request returned from the target terminal.

[0122] In one possible embodiment, the lost item notification device further comprises a fourth transmitting module used for at least one of: transmitting the repeated notification information to the target terminal in response to a time interval between the current time and the transmission time of the first notification information reaching a first predetermined time; transmitting the repeated notification information to the target terminal in response to a time interval between the current time and the reception time of the repeated notification request reaching a second predetermined time; transmitting the repeated notification information to the target terminal in response to a time interval between the current time and the transmission time of the first notification information reaching a first predetermined time and detecting that the lost item has not been taken out of the cabin; and transmitting the repeated notification information to the target terminal in response to a time interval between the current time and the reception time of the repeated notification request reaching a second predetermined time and detecting that the lost item has not been taken out of the cabin.

[0123] In one possible embodiment, the lost item notification device further includes a second control module for generating control information that controls an in-vehicle device installed in the cabin to output the repeated notification information in response to the target occupant returning the lost item to the cabin.

[0124] In one possible embodiment, the lost item notification device further comprises a first receiving module for receiving a first notification setting request via the vehicle's infotainment, and a first setting module for setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification based on the first notification setting request.

[0125] In one possible embodiment, the lost item notification device further includes a second receiving module for receiving a second notification setting request returned from the target terminal, and a second setting module for setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification based on the second notification setting request.

[0126] In an embodiment of the present disclosure, lost items are detected within the cabin of a vehicle, and upon detection of lost items, identity information of a target occupant who is the owner of the lost item is identified, and notification information is generated based on the identity information. This makes it possible to provide notification of lost items based on the identity information of the occupant who is the owner of the lost item within the cabin of the vehicle, and to provide notification of lost items in a target-oriented manner, thereby improving the effectiveness of lost item notifications and enhancing the property security of occupants.

[0127] In some embodiments, the functions or modules provided by the apparatuses provided in the embodiments of the present disclosure can be used to perform the methods described in the above method embodiments. For specific embodiments and technical effects, please refer to the description of the above method embodiments, and redundant description will be omitted here for brevity.

[0128] An embodiment of the present disclosure further provides a computer-readable storage medium having computer program instructions stored thereon, the computer-readable storage medium implementing the above method when executed by a processor, the computer-readable storage medium being a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0129] An embodiment of the present disclosure further provides a computer program product including computer readable code, the computer program product being operable, when executed in an electronic device, to cause a processor of the electronic device to execute instructions for implementing the above method.

[0130] An embodiment of the present disclosure further provides another computer program product for storing computer readable instructions that, when executed, cause a computer to perform operations of the lost item notification method according to any of the above embodiments.

[0131] An embodiment of the present disclosure further provides an electronic device including one or more processors and a memory storing executable instructions, the one or more processors configured to perform the above method by invoking the executable instructions stored in the memory.

[0132] The electronic device may be provided as a terminal, a server, or other form of device, for example, an infotainment or domain controller in the vehicle cabin, or a host for performing image or other data processing operations in an OMS or DMS.

[0133] 6 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0134] Referring to FIG. 6, electronic device 800 may include one or more of a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0135] The processing component 802 typically controls the overall operation of the electronic device 800, such as operations related to display, phone calls, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 that execute instructions to perform all or some of the steps of the above methods. The processing component 802 may also include one or more modules for interaction with other components. For example, the processing component 802 may include a multimedia module for interaction with the multimedia component 808.

[0136] The memory 804 is configured to store various types of data to support operation on the electronic device 800. These data include, by way of example, instructions for any application programs or methods operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by various types of volatile or non-volatile storage devices, such as, for example, static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks, or combinations thereof.

[0137] The power component 806 provides power to each component of the electronic device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0138] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, it may be implemented as a touch screen that receives input signals from a user. The touch panel may include one or more touch sensors to detect touches, slides, and gestures on the touch panel. The touch sensors may detect not only the boundaries of a touch or slide motion, but also the duration and pressure of the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a capture mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front and rear cameras may have a fixed optical lens system or a focal length and optical zoom capability.

[0139] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operational mode, such as a call mode, a record mode, or a voice recognition mode. The received audio signal may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting the audio signal.

[0140] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0141] The sensor component 814 includes one or more sensors for evaluating the state of each surface of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components such as the display and keypad of the electronic device 800, and the sensor component 814 can further detect changes in the position of the electronic device 800 or certain components of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration or deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor component 814 includes a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 814 may further include an optical sensor for use in imaging applications, such as a complementary metal oxide semiconductor (CMOS) or charge-coupled device (CCD) image sensor. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0142] The communication component 816 is configured to realize wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G) / LTE, a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be realized by radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0143] In an exemplary embodiment, electronic device 800 may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components and may be used to perform the methods described above.

[0144] In an exemplary embodiment, a non-volatile computer readable storage medium, such as a memory 804, is further provided that includes computer program instructions that, when executed by the processor 820 of the electronic device 800, can cause the above-described method to be performed.

[0145] FIG. 7 is another block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 7, the electronic device 1900 further includes a processing component 1922 including one or more processors, and a memory resource, represented by a memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. The processing component 1922 is also configured to perform the above method by executing the instructions.

[0146] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Microsoft Windows Server operating system (Windows Server™), Apple's graphical user interface operating system Mac OS X™, a multitasking, multiuser operating system for computers (Unix™), a free and open source Unix-based operating system (Linux™), an open source Unix-based operating system (FreeBSD™), or the like.

[0147] In an exemplary embodiment, a non-volatile computer-readable storage medium, such as a memory 1932 containing computer program instructions, which when executed by the processing component 1922 of the electronic device 1900, can cause the above-described method to be performed, is further provided.

[0148] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to implement aspects of the present disclosure.

[0149] A computer-readable storage medium may be a tangible device capable of storing and storing instructions for use in an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. Further specific examples (non-exhaustive list) of computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or slot-in protrusion structures on which instructions are stored, and any suitable combination of the above. As used herein, a computer-readable storage medium is not to be construed as being a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated via a waveguide or other transmission medium (e.g., pulsed light passing through a fiber optic cable), or electrical signals transmitted via electrical wires.

[0150] The computer readable program instructions described herein may be downloaded from a computer readable storage medium into each computing / processing device, or may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in a computer readable storage medium within each computing / processing device.

[0151] Computer program instructions for carrying out the operations of the present disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or target code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and general procedural programming languages ​​such as the "C" language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, state information of the computer readable program instructions may be used to personalize electronic circuitry, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), to execute the computer readable program instructions to implement aspects of the present disclosure.

[0152] Although various aspects of the present disclosure have been described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure, it should be understood that each block of the flowcharts and / or block diagrams, and combinations of each block in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0153] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus such that, when these instructions are executed by the processor of the computer or other programmable data processing apparatus, it produces an apparatus that performs the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may be stored on a computer readable storage medium to cause a computer, programmable data processing apparatus and / or other device to operate in a determined manner. Thus, a computer readable storage medium having instructions stored thereon includes an article of manufacture having instructions that implement each aspect of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0154] The computer readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device, and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer executable process, such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0155] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of systems, methods, and computer program products according to several embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or part of instructions, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions depicted in the blocks may be implemented in a different order than that depicted in the drawings. For example, two consecutive blocks may be substantially executed in parallel, or may be executed in a reverse order depending on the function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated system based on hardware that executes the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0156] The computer program product may be specifically implemented by hardware, software or a combination thereof. In one possible embodiment, the computer program product may be specifically a computer storage medium. In another possible embodiment, the computer program product may be specifically a software product, such as a Software Development Kit (SDK).

[0157] Although the embodiments of the present disclosure have been described above, the above description is merely illustrative and not exhaustive, and is not limited to the embodiments disclosed. Various modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms selected in this specification are intended to appropriately interpret the principles, practical applications, or technical improvements to the prior art of the embodiments, or to allow other skilled in the art to understand the embodiments disclosed in this specification.

[0158] This application claims priority to a Chinese patent application filed on August 31, 2020, with application number 202010896526.8, and entitled "Method, apparatus, electronic device and storage medium for notifying lost property," the entire contents of which are incorporated herein by reference.

Claims

1. By invoking instructions executable by one or more processors, Detecting lost property within a cabin of a vehicle; In response to the detection of the lost item, determining identity information of a target occupant who is an owner of the lost item; generating notification information based on the identity information; This includes performing the process of The step of generating notification information based on the identity information includes invoking instructions executable by one or more processors, generating first notification information based on the identity information; The method further includes, after generating notification information based on the identity information, calling an executable instruction using one or more processors to transmit the first notification information to a target terminal corresponding to the identity information; and / or The step of generating notification information based on the identity information includes: invoking instructions executable by one or more processors to generate second notification information based on the identity information and the information on the lost property; The method further includes, after generating notification information based on the identity information, performing a process of transmitting the second notification information to a management terminal connected to the vehicle and / or a management server connected to the vehicle by invoking an instruction executable by one or more processors; After sending the first notification information to the target terminal corresponding to the identity information, invoking instructions executable by one or more processors, The method further includes the steps of: generating repeat notification information in response to receiving a repeat notification request returned from the target terminal; by invoking instructions executable by one or more processors after generating the repeated notification information as described above; transmitting the repeated notification information to the target terminal in response to a time interval between a current time and a transmission time of the first notification information reaching a first predetermined time; transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of receiving the repeat notification request reaching a second predetermined time; transmitting the repeated notification information to the target terminal in response to detecting that a time interval between a current time and a transmission time of the first notification information reaches a first predetermined time and that the lost item has not been taken out of the cabin; and transmitting the repeated notification information to the target terminal in response to detecting that a time interval between a current time and a time of receiving the repeated notification request reaches a second predetermined time and that the lost item has not been removed from the cabin. and / or The method for notifying lost property further comprises, after generating the repeated notification information, calling up instructions executable by one or more processors, performing a process for generating control information that controls an in-vehicle device installed in the cabin to output the repeated notification information in response to the target occupant returning it to the cabin.

2. In response to detecting the lost item, determining an identity of a target occupant who is an owner of the lost item may include invoking instructions executable by one or more processors, responsive to the lost item being detected, determining a location of the lost item within the cabin; The method for notifying lost property as described in claim 1, further comprising the steps of: identifying identity information of a target occupant who is the owner of the lost property based on the location of the lost property within the cabin;

3. The step of determining an identity of a target occupant who is an owner of the lost item based on a location of the lost item within the cabin may include invoking instructions executable by one or more processors, the instructions comprising: determining an associated area of ​​the lost item within the cabin based on a location of the lost item within the cabin; performing face recognition on the relevant region based on the video stream in the cabin, and obtaining a face recognition result for the relevant region; The lost property notification method according to claim 2 , further comprising: identifying identity information of a target occupant who is the owner of the lost property based on the face recognition result of the related area.

4. The step of performing face recognition on the relevant region based on the video stream in the cabin and obtaining a face recognition result for the relevant region may include invoking instructions executable by one or more processors, determining a target video frame from the video stream based on a time when the lost item first appears in the video stream within the cabin; performing face recognition on a related region of the target video frame to obtain a face recognition result of the related region; and / or The step of identifying the identity of the target occupant who is the owner of the lost property based on the facial recognition result of the related area includes invoking instructions executable by one or more processors, determining candidate occupants for the lost property based on face recognition results of the relevant region; The method for notifying lost items as described in claim 3, further comprising the steps of: identifying the identity information of a target occupant who is the owner of the lost item from among the candidate occupants who entered the cabin at a time earlier than or the same time as the time the lost item first appeared in the video stream and who left the cabin at a time later than the time the lost item first appeared in the video stream.

5. In response to detecting the lost item, determining an identity of a target occupant who is an owner of the lost item may include invoking instructions executable by one or more processors, The lost item notification method of claim 1, further comprising, in response to detection of a lost item, performing a process of identifying identity information of a target occupant who is the owner of the lost item based on the time when the lost item first appeared in the video stream within the cabin.

6. The step of determining an identity of a target occupant who is an owner of the lost item based on a time when the lost item first appears in the in-cabin video stream may include invoking instructions executable by one or more processors, the instructions comprising: acquiring a usage history of the vehicle, including user identification information and corresponding usage time; Identifying a target user's identity information according to the usage history, the corresponding usage time being consistent with the time when the lost item first appears in the video stream; 6. The method for notifying lost property according to claim 5, further comprising the steps of: identifying the identity information of the target user as identity information of a target passenger who is the owner of the lost property.

7. By invoking instructions executable by one or more processors, capturing images and / or videos of occupants as they enter the cabin; Detecting information of an item carried by the occupant based on an image and / or a video when the occupant enters the cabin; The method further includes performing a process of associating information about an item carried by the occupant with identification information about the occupant; In response to detecting the lost item, determining an identity of a target occupant who is an owner of the lost item may include invoking instructions executable by one or more processors, A method for notifying lost property as described in any one of claims 1 to 6, characterized in that, in response to detection of a lost item, a process is performed to identify the identity information of a occupant associated with the lost item as the identity information of a target occupant who is the owner of the lost item.

8. The step of generating second notification information based on the identity information and the lost property information includes invoking instructions executable by one or more processors, The method for notifying lost property as described in claim 1, characterized in that it includes a process of generating second notification information based on the vehicle ID information, at least one of the boarding location of the target occupant and the disembarking location of the target occupant, the identity information and the lost property information.

9. generating notification information based on the identity information includes invoking instructions executable by one or more processors to generate third notification information based on the identity information and the lost property information if the target occupant is not a driver of the vehicle; the method of notifying lost property further includes performing a process of transmitting the third notification information to a terminal of a driver of the vehicle by invoking instructions executable by one or more processors; and / or The step of generating notification information based on the identity information includes: invoking instructions executable by one or more processors to generate fourth notification information based on the identity information; The lost item notification method according to any one of claims 1 to 8, further comprising: performing a process of generating control information for controlling an in-vehicle device installed in the cabin to output the fourth notification information by calling instructions executable by one or more processors.

10. By invoking instructions executable by one or more processors, receiving a first notification setting request by a vehicle device of the vehicle; The method further includes performing a process of setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification, based on the first notification setting request; and / or Invoking instructions executable by one or more processors; receiving a second notification setting request returned from the target terminal; The lost property notification method according to any one of claims 1 to 9, further comprising the step of setting at least one of an item type that does not require notification, an item type that requires notification, and an item type that requires repeated notification based on the second notification setting request.

11. a first detection module for detecting lost items within a cabin of the vehicle; an identification module that, in response to the detection of the lost item, identifies an identity of a target occupant who is an owner of the lost item; a first generating module for generating notification information based on the identity information; one or more processors that invoke executable instructions to be used as The first generating module further generates first notification information based on the identity information; The one or more processors call executable instructions, and are further used as a sending module for sending the first notification information to a target terminal corresponding to the identity information; and / or The first generating module further generates second notification information based on the identity information and the lost property information; The one or more processors call executable instructions, and are further used as a transmission module for transmitting the second notification information to a management terminal connected to the vehicle and / or a management server connected to the vehicle; The one or more processors invoke executable instructions and are further used as a second generating module for generating repeat notification information in response to receiving a repeat notification request returned from the target terminal; The transmission module further comprises: transmitting the repeated notification information to the target terminal in response to a time interval between a current time and a transmission time of the first notification information reaching a first predetermined time; transmitting the repeat notification information to the target terminal in response to a time interval between a current time and a time of receiving the repeat notification request reaching a second predetermined time; transmitting the repeated notification information to the target terminal in response to detecting that a time interval between a current time and a transmission time of the first notification information reaches a first predetermined time and that the lost item has not been taken out of the cabin; transmitting the repeated notification information to the target terminal in response to detecting that a time interval between a current time and a time of receiving the repeated notification request reaches a second predetermined time and that the lost item has not been removed from the cabin; and / or The lost item notification device is characterized in that the one or more processors are used as a third generation module that calls executable instructions and further generates control information that controls an in-vehicle device installed in the cabin to output the repeated notification information in response to the target occupant returning it to the cabin.

12. one or more processors; a storage memory that stores executable instructions, The one or more processors are configured to execute the lost item notification method of any one of claims 1 to 10 by calling executable instructions stored in the storage memory.

13. A computer-readable storage medium having stored thereon computer program instructions, the computer-readable storage medium being characterized in that, when executed by a processor, the computer program instructions cause a lost property notification method according to any one of claims 1 to 10 to be realized.

14. A computer program comprising computer-readable code, the computer-readable code being executed in an electronic device to cause a processor of the electronic device to execute instructions for implementing the lost property notification method according to any one of claims 1 to 10.

Images