Article anti-loss method and device, vehicle, computer equipment and storage medium

Abstract

The invention relates to an article anti-loss method and device, a vehicle, computer equipment, a storage medium and a computer program product. The method comprises the following steps: when a vehicle arrives at a departure place, acquiring a first image group of a getting-on process of a passenger carrying a to-be-identified article through a vehicle-mounted camera; performing article feature recognition on the first image group to obtain an article list of the to-be-recognized articles; when the vehicle arrives at the destination, acquiring a second image group of the getting-off process of the passenger carrying the to-be-identified article through a vehicle-mounted camera; performing article feature recognition on the second image group to obtain a recognition result; and if the identification result shows that the passenger does not carry all the articles in the article list, generating first voice prompt information to remind the passenger to check whether the articles are missed or not. By adopting the method, the problem of reminding failure caused by reminding only when the passenger leaves can be avoided, and the real-time performance and efficiency of reminding are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a method, device, vehicle, computer device, storage medium, and computer program product for preventing loss of items. Background Art

[0002] With the popularization of driverless vehicle technology, the scenario of passengers taking driverless operating vehicles is increasing. When passengers get off the vehicle, they are prone to leaving items in the vehicle due to carelessness and without driver reminder.

[0003] For the ordinary function of reminding to prevent loss of items, a fixed voice reminder is automatically broadcast after the vehicle arrives at the destination. When an item is actually left in the vehicle, the vehicle cannot timely and accurately feedback information to the passenger. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, vehicle, computer device, storage medium, and computer program product for preventing loss of items that can improve the reminder efficiency.

[0005] In a first aspect, this application provides a method for preventing loss of items, the method including:

[0006] When the vehicle arrives at the departure place, obtain a first image group of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera;

[0007] Perform item feature recognition on the first image group to obtain an item list of the item to be recognized;

[0008] When the vehicle arrives at the destination, obtain a second image group of the process of the passenger carrying the item to be recognized getting off the vehicle through the in-vehicle camera;

[0009] Perform item feature recognition on the second image group to obtain a recognition result;

[0010] If the recognition result indicates that the passenger does not carry all the items in the item list, generate a first voice prompt message to remind the passenger to check whether there is any item left.

[0011] In one embodiment, the in-vehicle camera includes an in-vehicle camera and a rear camera, the first image group includes multiple consecutive first in-vehicle images and / or first trunk images, and when the vehicle arrives at the departure place, obtaining a first image group of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera includes:

[0012] When the vehicle arrives at the departure place and the vehicle door is in an open state, photograph the seat area of the vehicle through the in-vehicle camera to obtain multiple consecutive first in-vehicle images;

[0013] When the tailgate of the vehicle is in the open state, the trunk of the vehicle is photographed by the rear camera to obtain a plurality of consecutive first trunk images.

[0014] In one embodiment, the item feature recognition is performed on the first image group to obtain an item list of the item to be recognized, including:

[0015] For each image in the first image group, a first bounding box corresponding to the item to be recognized carried by the passenger in the image is marked;

[0016] Based on the positions of the first bounding boxes of the items to be recognized in each image of the first image group, a first movement trajectory of the items to be recognized is determined;

[0017] If the first movement trajectory indicates that the item to be recognized moves from a preset out-of-vehicle area to a preset in-vehicle area, it is determined that the passenger moves the item to be recognized from outside the vehicle to inside the vehicle, and the types and quantities of the items to be recognized in the in-vehicle area are identified;

[0018] Based on the types and quantities of the items to be recognized in the in-vehicle area, an item list of the items to be recognized is determined.

[0019] In one embodiment, the item feature recognition is performed on the second image group to obtain a recognition result, including:

[0020] For each image in the second image group, a second bounding box corresponding to the passenger in the image is marked;

[0021] Based on the positions of the second bounding boxes corresponding to the passenger in each image of the second image group, a second movement trajectory of the passenger is determined;

[0022] If the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, it is determined that the passenger has got out of the vehicle, and the types and quantities of the items to be recognized in the in-vehicle area are identified;

[0023] Based on the types and quantities of the items to be recognized in the in-vehicle area, a recognition result is determined.

[0024] In one embodiment, the determining the recognition result based on the types and quantities of the items to be recognized in the in-vehicle area includes:

[0025] For each image in the second image group, a third bounding box corresponding to the item to be recognized in the image is marked;

[0026] Determine the third motion trajectory of the item to be recognized based on the positions of the third bounding boxes corresponding to the item to be recognized in each image of the second image group;

[0027] If the third motion trajectory indicates that the item to be recognized is in the vehicle area, determine that the recognition result indicates that the passenger does not carry all the items in the item list.

[0028] In one embodiment, the method further includes:

[0029] If the third motion trajectory indicates that the item to be recognized is in the out-of-vehicle area, determine that the recognition result indicates that the passenger carries all the items in the item list.

[0030] In one embodiment, the first image group includes first in-vehicle images, the second image group includes second in-vehicle images, and the method further includes:

[0031] Perform item feature recognition on the first in-vehicle images to obtain a first list of inherent items of the vehicle;

[0032] Perform item feature recognition on the second in-vehicle images to obtain a second list of inherent items of the vehicle;

[0033] If the types and / or quantities of items in the second list of inherent items are inconsistent with those in the first list of inherent items, determine that the passenger carries the inherent items of the vehicle, and generate a second voice prompt message to remind the passenger to return the inherent items.

[0034] In one embodiment, the method further includes:

[0035] Obtain a payment order;

[0036] After the payment order is completed, generate a feedback message based on the first voice prompt message and the second voice prompt message, and feedback the feedback message to the mobile device held by the passenger.

[0037] In a second aspect, the present application further provides an item anti-loss device. The device includes:

[0038] A shooting module, configured to obtain a first image group of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera when the vehicle arrives at the departure place;

[0039] An identification module, configured to perform item feature recognition on the first image group to obtain an item list of the item to be recognized;

[0040] The shooting module is further configured to, when the vehicle arrives at the destination, obtain a second image set of the process of the passenger getting off the vehicle while carrying the item to be recognized through the vehicle-mounted camera;

[0041] The recognition module is further configured to perform item feature recognition on the second image set to obtain a recognition result;

[0042] The reminder module is configured to, if the recognition result indicates that the passenger does not carry all the items in the item list, generate a first voice prompt message to remind the passenger to check whether there is any item omission.

[0043] In a third aspect, the present application further provides a vehicle, which includes a display unit, and the display unit implements the steps of the above item anti-loss method.

[0044] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above item anti-loss method are implemented.

[0045] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the above item anti-loss method are implemented.

[0046] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above item anti-loss method are implemented.

[0047] For the above item anti-loss method, device, vehicle, computer device, storage medium and computer program product, by obtaining the first image set and the second image set respectively when the vehicle arrives at the departure place and the destination, and immediately performing item feature recognition, an item list of the item to be recognized can be quickly generated when the passenger gets on the vehicle with the item to be recognized, and a recognition result can be quickly generated when the passenger gets off the vehicle. This real-time item recognition and list generation can greatly improve the item management efficiency of passengers and reduce the possibility of omission or loss. In addition, when the recognition result indicates that the passenger does not carry all the items in the item list, a first voice prompt message is generated to remind the passenger to check whether there is any item omission, and the passenger is timely reminded of leaving the item to be recognized in the vehicle. This real-time reminder can avoid the problem that the reminder fails because the reminder is made after the passenger has left, and improves the real-time performance and efficiency of the reminder. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is an application environment diagram of the item anti-loss method in an embodiment;

[0049] Figure 2 Schematic flowchart of an article anti-loss method in an embodiment;

[0050] Figure 3 Schematic structural diagram of a vehicle in an embodiment;

[0051] Figure 4 Block diagram of the structure of an article anti-loss device in an embodiment;

[0052] Figure 5 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The article anti-loss method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . After the vehicle arrives at the departure place, the intelligent antenna module (TCAM) 102 transmits the arrival at the departure place signal (ArvngStartng = ON) to the entertainment host (DHU) 104. After receiving the departure place signal, the entertainment host 104 obtains a first image group of the process of passengers carrying the articles to be identified getting on the vehicle through the on-vehicle camera; the entertainment host 104 performs article feature recognition on the first image group to obtain an article list of the articles to be identified; when the vehicle arrives at the destination, the intelligent antenna module 102 sends the arrival at the destination signal (ArvngDestination = ON) to the entertainment host 104. After receiving the destination signal, the entertainment host 104 obtains a second image group of the process of passengers carrying the articles to be identified getting off the vehicle through the on-vehicle camera; the entertainment host 104 performs article feature recognition on the second image group to obtain an identification result; if the identification result indicates that the passengers do not carry all the articles in the article list, the entertainment host 104 generates a first voice prompt message to remind the passengers to check whether there are any missing articles.

[0055] Among them, both the intelligent antenna module 102 and the entertainment host 104 are components of the vehicle. The intelligent antenna module 102 integrates a high-precision GNSS (Global Navigation Satellite System) antenna, which can receive signals from multiple satellite systems and provide centimeter-level positioning information for the vehicle. The entertainment host 104, as the driving information display center, displays various information of the vehicle, such as vehicle speed, engine speed, fuel quantity, water temperature, etc., in the form of graphics or numbers to the driver. Secondly, the entertainment host 104 also serves as an entertainment center to provide functions such as audio and video playback. In addition, the entertainment host 104 also integrates a navigation function to provide real-time route navigation information for the driver.

[0056] In one embodiment, as Figure 2 shown, an article anti-loss method is provided. Taking the entertainment host in Figure 1 as an example for illustration, the method includes the following steps:

[0057] Step 202, when the vehicle arrives at the departure place, obtain a first image group of the process of a passenger carrying an article to be identified getting on the vehicle through an in-vehicle camera.

[0058] Among them, the departure place refers to the place where the passenger starts to take the vehicle.

[0059] The in-vehicle camera refers to a camera installed inside or outside the vehicle, which is used to capture the environment around the vehicle or the behavior of the passengers. For example, the in-vehicle camera includes an in-car camera and a rear camera. Among them, the in-car camera is used to capture images of the co-pilot, the rear seats, and the floor area, and the rear camera is used to capture images of the trunk area.

[0060] The article to be identified refers to an article that needs to be identified and classified, such as luggage, packages, etc.

[0061] The first image group refers to a series of images of the process of a passenger carrying an article to be identified getting on the vehicle captured by the in-vehicle camera. Among them, the process of a passenger carrying an article to be identified getting on the vehicle includes the process of a passenger carrying an article to be identified entering the co-pilot or the rear seats, and the process of a passenger storing the article to be identified in the trunk. Therefore, the first image group includes multiple consecutive first in-car images and / or first trunk images. Among them, the first in-car images refer to images of the co-pilot, the rear seats, and the floor area, and the first trunk images refer to images of the trunk area. If the passenger only carries an article to be identified and enters the co-pilot or the rear seats, the first image group includes first in-car images; if the passenger stores the article to be identified in the trunk and then enters the co-pilot or the rear seats, the first image group includes first in-car images and first trunk images; if the passenger only stores the article to be identified in the trunk, the first image group includes first trunk images.

[0062] Specifically, when the vehicle arrives at the departure place, the intelligent antenna module (TCAM) transmits the arrival at the departure place signal (ArvngStartng = ON) to the entertainment host (DHU). After receiving the arrival at the departure place signal, the entertainment host controls the in-vehicle camera to start working, captures the process of a passenger carrying an article to be identified getting on the vehicle, obtains the first image group, and the in-vehicle camera transmits the captured first image group to the entertainment host. The entertainment host uses image recognition technology to identify and classify the first image group. For example, deep learning algorithms can be used to extract features and classify the images to identify the article to be identified.

[0063] Step 204: Perform item feature recognition on the first image group to obtain an item list of the item to be recognized.

[0064] Among them, item feature recognition refers to the process of identifying and classifying items by analyzing item features in the image, such as color, shape, size, texture, etc. For example, use deep learning algorithms to extract and classify features of the image to identify the item to be recognized.

[0065] The item list refers to a list of the items to be recognized, including information such as the name, quantity, and size of the items.

[0066] Specifically, the in-vehicle camera transmits the first image group to the entertainment host. The entertainment host uses image processing software or algorithms to preprocess the first image group, such as adjusting the image size, contrast, brightness, etc., to improve the recognition accuracy. The entertainment host uses item feature recognition algorithms to analyze the preprocessed images, extract the features of the items to be recognized. The entertainment host uses a classifier to classify and identify the items based on the extracted features. The classifier can be trained based on deep learning models or other machine learning algorithms to identify different types of items. The entertainment host generates an item list of the items to be recognized according to the classification results, including information such as the name, quantity, and size of the items.

[0067] In some embodiments, if the type of the item to be recognized is a prohibited item or a dangerous item, a voice prompt is directly given to remind the driver to prohibit dangerous items from getting on the vehicle.

[0068] Step 206: When the vehicle arrives at the destination, obtain a second image group of the process of the passenger getting off the vehicle with the item to be recognized through the in-vehicle camera.

[0069] Among them, the destination refers to the place where the passenger arrives by car.

[0070] The second image group refers to a series of images of the process of the passenger getting off the vehicle with the item to be recognized captured by the in-vehicle camera. Among them, the process of the passenger getting off the vehicle with the item to be recognized includes the process of the passenger moving the item to be recognized from the co-pilot or the rear seat to outside the vehicle, and the process of the passenger moving the item to be recognized from the trunk to outside the vehicle. Therefore, the second image group includes multiple consecutive second in-vehicle images and / or second trunk images. Among them, the second in-vehicle images refer to images of the co-pilot, the rear seat, and the floor area, and the second trunk images refer to images of the trunk area. If the passenger only moves the item to be recognized from the co-pilot or the rear seat to outside the vehicle, the first image group includes second in-vehicle images; if the passenger moves the item to be recognized from the co-pilot or the rear seat to outside the vehicle and then moves the item to be recognized from the trunk to outside the vehicle, the second image group includes second in-vehicle images and second trunk images.

[0071] Specifically, when the vehicle arrives at the destination, the intelligent antenna module (TCAM) transmits the arrival destination signal (ArvngDestination = ON) to the entertainment host unit (DHU). After receiving the arrival destination signal, the in-vehicle camera starts to work and captures the process of passengers getting off the vehicle with the items to be identified, obtaining the second image group. The in-vehicle camera transmits the first image group captured to the entertainment host unit, and the entertainment host unit uses image recognition technology to identify and classify the second image group. For example, deep learning algorithms can be used to extract features and classify the images to identify the items to be identified.

[0072] Step 208: Conduct item feature recognition on the second image group to obtain the recognition result.

[0073] Among them, the recognition result includes a list of items to be identified left in the vehicle (such as the co-pilot seat, rear seats, floor, or trunk area).

[0074] Specifically, the entertainment host unit uses image processing software or algorithms to preprocess the second image group, such as adjusting the image size, contrast, brightness, etc., to improve the recognition accuracy. The entertainment host unit uses item feature recognition algorithms to analyze the preprocessed images, extracts the features of the items to be identified left in the co-pilot seat, rear seats, floor, or trunk area. The entertainment host unit classifies and identifies the items based on the extracted features using a classifier, obtaining a list of items to be identified left in the co-pilot seat, rear seats, floor, or trunk area. Among them, the classifier can be trained based on deep learning models or other machine learning algorithms to identify different types of items.

[0075] Step 210: If the recognition result indicates that the passenger does not carry all the items in the item list, generate a first voice prompt message to remind the passenger to check for any missing items.

[0076] Among them, if there are items to be identified in the list of items to be identified left in the co-pilot seat, rear seats, floor, or trunk area, it is determined that the recognition result indicates that the passenger does not carry all the items in the item list; if there are no items to be identified in the list of items to be identified left in the co-pilot seat, rear seats, floor, or trunk area, it is determined that the recognition result indicates that the passenger carries all the items in the item list.

[0077] The first voice prompt message is broadcast through the vehicle's speaker. The first semantic prompt message is used to remind the passenger to check for any missing items. Among them, the first voice prompt message can be "Please note if there are any items left in the vehicle".

[0078] In some embodiments, if the recognition result indicates that the passenger has not carried all the items in the item list, the first voice prompt information is generated according to the names and quantities of the items to be recognized left in the vehicle in the recognition result, and the first voice prompt information is used to remind the passenger of the specific names and quantities of the items to be recognized that are left behind.

[0079] For example, the first voice prompt information can be "Please take the luggage in the trunk" and so on.

[0080] Specifically, the entertainment host compares the recognition result with the item list of the items to be recognized carried by the passenger. If any item in the item list of the items to be recognized is included in the recognition result, it means that the passenger may have left some items behind. The entertainment host generates a first voice prompt information and plays it through the in-vehicle audio system to remind the passenger to check if there are any items left behind.

[0081] In the above item anti-loss method, by obtaining the first image group and the second image group respectively when the vehicle arrives at the departure place and the destination and immediately performing item feature recognition, the item list of the items to be recognized can be quickly generated when the passenger gets on the vehicle with the items to be recognized, and the recognition result can be quickly generated when the passenger gets off the vehicle. This real-time item recognition and list generation can greatly improve the item management efficiency of the passenger and reduce the possibility of omission or loss. In addition, when the recognition result indicates that the passenger has not carried all the items in the item list, the first voice prompt information is generated to remind the passenger to check if there are any items left behind and timely remind the passenger of the items to be recognized left in the vehicle. This real-time reminder can avoid the problem that the reminder fails because the reminder is made after the passenger has left, and improves the real-time performance and efficiency of the reminder.

[0082] In one embodiment, as can be seen from the above embodiment, the in-vehicle camera includes an in-vehicle camera and a rear camera, and the first image group includes multiple consecutive first in-vehicle images and / or first trunk images. Therefore, when the vehicle arrives at the departure place, the steps of obtaining the first image group of the process of the passenger getting on the vehicle with the items to be recognized through the in-vehicle camera are as follows:

[0083] When the vehicle arrives at the departure place and the vehicle door is in the open state, the in-vehicle camera takes pictures of the seat area of the vehicle to obtain multiple consecutive first in-vehicle images; when the tailgate of the vehicle is in the open state, the rear camera takes pictures of the trunk of the vehicle to obtain multiple consecutive first trunk images.

[0084] Among them, the in-vehicle camera is installed inside the vehicle and is used to take pictures of the in-vehicle environment.

[0085] The seat area refers to the areas of the co-pilot, the rear seats and the floor.

[0086] The first in-vehicle image refers to multiple consecutive images of the vehicle seat area captured by an in-vehicle camera.

[0087] The tailgate refers to the luggage compartment lid of the vehicle. For example, the trunk lid or the front trunk lid.

[0088] The trunk refers to the rear storage compartment of the vehicle, which is usually used to store items carried by passengers. For example, the trunk refers to the rear trunk.

[0089] The rear camera is installed at the rear of the vehicle and is used to capture the rear environment of the vehicle.

[0090] The first trunk image refers to multiple consecutive images of the vehicle trunk captured by the rear camera.

[0091] Specifically, when the vehicle arrives at the departure location, the intelligent antenna module (TCAM) transmits the arrival at departure signal (ArvngStartng = ON) to the entertainment host unit (DHU). After receiving the departure signal, the entertainment host unit obtains the driving state signal and the vehicle speed of the vehicle. If the driving state signal indicates that the current gear of the vehicle is in the P gear, the vehicle speed is 0, and it receives the tailgate opening signal (Trsts = ON) sent by the tailgate controller, then the entertainment host unit sends an image acquisition signal (PhotoShot = ON) to the rear camera every 0.2 s switching cycle. The rear camera continuously captures the trunk of the vehicle, generates multiple consecutive first trunk images, and sends a feedback signal (StsOfPhotoShotTrunk = ON) to the entertainment host unit, and transmits the first trunk images to the entertainment host unit. The entertainment host unit determines that there are items to be recognized stored in the trunk based on the first trunk images and generates an item storage signal, which is used to indicate that there are items to be recognized stored in the trunk. When the passengers get off the vehicle, the item storage signal can be used to remind the passengers to carry the items to be recognized in the trunk. After the entertainment host unit receives the departure signal and determines through the sensor that the current vehicle door has switched from the closed state to the open state, it sends a control command (PhotoShot = ON) to the in-vehicle camera to instruct the in-vehicle camera to start working. The in-vehicle camera continuously captures the seat area of the vehicle, generates multiple consecutive first in-vehicle images, and the in-vehicle camera sends a feedback message (StsOfPhotoShot = ON) to the entertainment host unit. After receiving the feedback message, the entertainment host unit starts to receive the first in-vehicle images captured by the in-vehicle camera.

[0092] In some embodiments, Figure 3 is a schematic structural diagram of a vehicle in an embodiment, as Figure 3As shown in the figure, the vehicle includes an intelligent antenna module (TCAM), an entertainment host (DHU), a rear camera, an in-vehicle camera, a tailgate controller, and a speaker. Among them, the tailgate controller is used to control the opening and closing of the vehicle's tailgate and feedback the state of the tailgate to the entertainment host.

[0093] In this embodiment, when the vehicle arrives at the departure place and the vehicle's doors are in the open state, multiple consecutive first in-vehicle images are immediately acquired, and when the vehicle's tailgate is in the open state, multiple consecutive first trunk images are immediately acquired. In other words, in this embodiment, when the passenger opens the door or the tailgate, the shooting and item feature recognition start, and a list of items to be recognized can be quickly generated. When giving reminders later, reference data can be provided in a timely manner, thereby improving the real-time performance and efficiency of the reminders.

[0094] In one embodiment, item feature recognition is performed on the first image group to obtain a list of items to be recognized, including the following steps:

[0095] First, for each image in the first image group, a first bounding box corresponding to the item to be recognized carried by the passenger in the targeted image is marked.

[0096] Among them, each image in the first image group refers to the first in-vehicle image and the first trunk image.

[0097] The first bounding box refers to a rectangular box marked on each first in-vehicle image and first trunk image in the first image group, which is used to indicate the position of the item to be recognized carried by the passenger in the image.

[0098] Specifically, the entertainment host preprocesses each first in-vehicle image and first trunk image in the first image group, such as adjusting the image size, contrast, brightness, etc., to improve the recognition accuracy. The entertainment host uses image processing technologies, such as edge detection, threshold processing, etc., to process each image to highlight the outline and boundary of the item to be recognized. Based on the outline and boundary of the item to be recognized, the entertainment host uses computer vision technologies, such as object detection algorithms, and automatically marks a rectangular box, that is, the first bounding box, on each first in-vehicle image and first trunk image through an automatic annotation algorithm to obtain the position of the item to be recognized in the seat area and / or the trunk.

[0099] Second, based on the positions of the first bounding boxes of the items to be recognized in each image in the first image group, the first movement trajectory of the items to be recognized is determined.

[0100] Among them, the first motion trajectory refers to the motion trajectory that describes the object to be recognized between each first in-vehicle image and / or first trunk image in the first image group. The first motion trajectory is determined based on the positions of the first bounding boxes in multiple consecutive images in the first image group. Therefore, the first motion trajectory includes the motion trajectory of the object to be recognized inside the vehicle and / or the motion trajectory of the object to be recognized inside the trunk. According to the first motion trajectory of the object to be recognized, it can be detected whether there is an action of moving an object into the vehicle.

[0101] Specifically, the entertainment host obtains the positions of the first bounding boxes of the object to be recognized in each image in the first image group, and determines the first motion trajectory of the object to be recognized according to the positions of the first bounding boxes in each image, using a trajectory generation algorithm or a motion model.

[0102] Third, when the first motion trajectory indicates that the object to be recognized moves from a preset out-of-vehicle area to a preset in-vehicle area, it is determined that the passenger moves the object to be recognized from outside the vehicle into the vehicle, and the types and quantities of the objects to be recognized in the in-vehicle area are identified.

[0103] Among them, the out-of-vehicle area refers to a preset area outside the vehicle, usually a relatively large spatial range. Usually, it is determined that the object to be recognized is in the preset out-of-vehicle area by the fact that the image captured by the out-of-vehicle camera contains the object to be recognized.

[0104] The in-vehicle area refers to a preset area inside the vehicle, usually the positions where passengers carry items when riding, such as the seat area, storage box, etc. Usually, it is determined that the object to be recognized is in the preset in-vehicle area by the fact that the image captured by the in-vehicle camera contains the object to be recognized.

[0105] Specifically, if the entertainment host compares the starting and ending positions of the first motion trajectory in the first image group and determines that the object to be recognized moves from the preset out-of-vehicle area to the preset in-vehicle area, it is determined that the passenger moves the object to be recognized from outside the vehicle into the vehicle. The entertainment host uses image processing technologies, such as color segmentation, feature extraction, etc., to identify different objects to be recognized and count their quantities.

[0106] Fourth, based on the types and quantities of the objects to be recognized in the in-vehicle area, an item list of the objects to be recognized is determined.

[0107] Specifically, the entertainment host processes and analyzes the images in the in-vehicle area to identify different objects to be recognized. The entertainment host counts the types and quantities of each object to be recognized according to the recognition results. The entertainment host combines the types and quantities of the recognized objects to be recognized to generate an item list of the objects to be recognized.

[0108] In this embodiment, by marking the first bounding box of the item to be recognized in the first image group and determining the first motion trajectory, the movement trajectory of the item to be recognized carried by the passenger in the vehicle can be accurately tracked. The movement trajectory of the item to be recognized in the vehicle can accurately reflect that the item to be recognized has entered the vehicle. Moreover, by using multiple consecutive images to recognize the type and quantity of the item to be recognized instead of only using one image of the passenger entering the vehicle, the probability of missed recognition and misrecognition of the item to be recognized is reduced, thereby improving the recognition accuracy of the item to be recognized.

[0109] In one embodiment, the second image group includes multiple consecutive second in-vehicle images and / or second trunk images. Therefore, when the vehicle reaches the destination, the second image group of the process of the passenger getting off the vehicle with the item to be recognized is obtained through the on-vehicle camera, including:

[0110] When the vehicle reaches the destination and the vehicle door is in the open state, the seat area of the vehicle is photographed by the in-vehicle camera to obtain multiple consecutive second in-vehicle images; when the tailgate of the vehicle is in the open state, the trunk of the vehicle is photographed by the rear camera to obtain multiple consecutive second trunk images.

[0111] Specifically, when the vehicle reaches the destination, the intelligent antenna module (TCAM) transmits the arrival destination signal (ArvngDestination = ON) to the entertainment host (DHU). After the entertainment host receives the departure signal and determines through the sensor that the current vehicle door has switched from the closed state to the open state, it sends a control instruction (PhotoShot = ON) to the in-vehicle camera to instruct the in-vehicle camera to start working. The in-vehicle camera continuously photographs the seat area of the vehicle to generate multiple consecutive second in-vehicle images. The in-vehicle camera sends a feedback message (StsOfPhotoShot = ON) to the entertainment host. After receiving the feedback message, the entertainment host starts to receive the second in-vehicle images photographed by the in-vehicle camera. When the entertainment host receives the destination signal, the second in-vehicle images do not contain passengers, and the state of the tailgate is in the open state, it sends an image acquisition signal (PhotoShot = ON) to the rear camera every 0.2 s switching cycle. The rear camera continuously photographs the trunk of the vehicle to generate multiple consecutive first trunk images, and sends a feedback signal (StsOfPhotoShotTrunk = ON) to the entertainment host, and transmits the first trunk images to the entertainment host.

[0112] In this embodiment, when the vehicle arrives at the destination and the vehicle door is in the open state, multiple consecutive second in-vehicle images are immediately acquired, and when the vehicle's tailgate is in the open state, multiple consecutive second trunk images are immediately acquired. In other words, in this embodiment, when the passenger opens the vehicle door or the tailgate, the shooting and item feature recognition start, and a list of items to be recognized can be quickly generated, and reference data can be provided in a timely manner during subsequent reminders, thereby improving the real-time performance and efficiency of the reminders.

[0113] In one embodiment, item feature recognition is performed on the second image group to obtain a recognition result, including:

[0114] First, for each image in the second image group, a second bounding box corresponding to the passenger in the targeted image is marked.

[0115] Among them, each image in the second image group refers to the second in-vehicle image and the second trunk image.

[0116] The second bounding box refers to a rectangular box marked on each image in the second image group, which is used to indicate the position of the passenger in the image.

[0117] Specifically, the entertainment host preprocesses each image in the second image group, such as adjusting the image size, contrast, brightness, etc., to improve the recognition accuracy. The entertainment host uses image processing techniques, such as edge detection and threshold processing, to process each image to highlight the contour and boundary of the passenger. The entertainment host uses computer vision techniques, such as object detection algorithms, and automatically marks a rectangular box, that is, the second bounding box, on each image through an automatic annotation algorithm according to the contour and boundary of the item to be recognized.

[0118] Second, based on the position of the second bounding box corresponding to the passenger in each image in the second image group, the second movement trajectory of the passenger is determined.

[0119] Among them, the second movement trajectory refers to the movement trajectory of the passenger between each image in the second image group. The second movement trajectory is determined according to the positions of the second bounding boxes in multiple consecutive images in the second image group. According to the second movement trajectory of the passenger, it can be detected whether the passenger gets out of the vehicle.

[0120] Specifically, the entertainment host obtains the position of the second bounding box of the passenger in each image in the second image group, and uses a trajectory generation algorithm or a motion model to determine the second movement trajectory of the passenger according to the position of the second bounding box in each image.

[0121] Third, if the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, it is determined that the passenger has got out of the vehicle, and the types and quantities of the items to be recognized in the in-vehicle area are recognized.

[0122] Specifically, if the entertainment host compares the starting and ending positions of the second movement trajectory in the second image group and determines that the passenger moves from a preset outside-vehicle area to a preset inside-vehicle area, it is determined that the passenger gets off the vehicle. The entertainment host uses image processing technologies, such as color segmentation and feature extraction, to identify different items to be recognized in the second image group and counts their quantities.

[0123] IV. Determine the recognition result based on the types and quantities of the items to be recognized in the inside-vehicle area.

[0124] Specifically, the entertainment host processes and analyzes the images of the inside-vehicle area to identify different items to be recognized. The entertainment host counts the types and quantities of each item to be recognized according to the recognition results. The entertainment host combines the types and quantities of the identified items to be recognized to determine the recognition result.

[0125] In this embodiment, by annotating the second bounding boxes of the items to be recognized in the second image group and determining the second movement trajectory, the movement trajectory of the passenger in the vehicle can be accurately tracked. The fact that the passenger has got off the vehicle can be accurately reflected by the movement trajectory of the passenger in the vehicle. Moreover, by recognizing the types and quantities of the items to be recognized in the inside-vehicle area through multiple consecutive images instead of only through a single inside-vehicle image, the probability of missed recognition and misrecognition of the items to be recognized is reduced, thereby improving the recognition accuracy of the items to be recognized.

[0126] In one embodiment, determining the recognition result based on the types and quantities of the items to be recognized in the inside-vehicle area includes the following steps:

[0127] I. For each image in the second image group, annotate the third bounding box corresponding to the item to be recognized in the targeted image.

[0128] The third bounding box refers to a rectangular box annotated on each image in the second image group, which is used to indicate the position of the item to be recognized carried by the passenger in the image.

[0129] Specifically, the entertainment host preprocesses each image in the second image group, such as adjusting the image size, contrast, brightness, etc., to improve the recognition accuracy. The entertainment host uses image processing technologies, such as edge detection and threshold processing, to process each image to highlight the contours and boundaries of the items to be recognized. Based on the contours and boundaries of the items to be recognized, the entertainment host uses computer vision technologies, such as object detection algorithms, and automatically annotates a rectangular box, i.e., the third bounding box, on each image through an automatic annotation algorithm.

[0130] II. Based on the positions of the third bounding boxes corresponding to the items to be recognized in each image in the second image group, determine the third movement trajectory of the items to be recognized.

[0131] Among them, the third motion trajectory refers to the motion trajectory of the item to be recognized between each image in the second image group. The third motion trajectory is determined based on the positions of the third bounding boxes in multiple consecutive images in the second image group. Based on the third motion trajectory of the item to be recognized, it is possible to detect whether there is an action of an item moving out of the vehicle.

[0132] Specifically, the entertainment host obtains the positions of the third bounding boxes of the item to be recognized in each image in the second image group, and based on the positions of the third bounding boxes in each image, uses a trajectory generation algorithm or a motion model to determine the third motion trajectory of the item to be recognized.

[0133] Third, if the third motion trajectory indicates that the item to be recognized is in the vehicle area, it is determined that the recognition result represents all the items in the list of items not carried by the passenger; if the third motion trajectory indicates that the item to be recognized is in the area outside the vehicle, it is determined that the recognition result represents all the items in the list of items carried by the passenger.

[0134] Among them, if the second motion trajectory indicates that the passenger moves from a preset vehicle area to a preset area outside the vehicle, and the third motion trajectory indicates that the item to be recognized is in the vehicle area, it indicates that the passenger is separated from the item to be recognized, and the passenger moves out of the vehicle. It is determined that the recognition result represents all the items in the list of items not carried by the passenger, and a first voice prompt message is generated. For example, the voice prompt is "Please note whether you have left any items in the vehicle".

[0135] If the second motion trajectory indicates that the passenger moves from a preset vehicle area to a preset area outside the vehicle, and the third motion trajectory indicates that the item to be recognized is in the area outside the vehicle, it indicates that the passenger and the item to be recognized move out of the vehicle together, and it is determined that the recognition result represents all the items in the list of items carried by the passenger.

[0136] Specifically, if the entertainment host compares the starting and ending positions of the third motion trajectory in the second image group, determines that the item to be recognized is in the preset vehicle area, and determines that the recognition result represents all the items in the list of items not carried by the passenger. If the entertainment host compares the starting and ending positions of the third motion trajectory in the second image group and determines that the item to be recognized moves from the preset vehicle area to the preset area outside the vehicle, it is determined that the passenger moves the item to be recognized from inside the vehicle to outside the vehicle, and the recognition result represents all the items in the list of items carried by the passenger.

[0137] In some embodiments, if the entertainment host receives a destination signal and the tailgate state switches from the closed state to the open state (Trsts = CLOSE to OPEN), the entertainment host sends an image acquisition signal (PhotoShot = ON) to the rear camera every 0.2 s switching cycle. The rear camera continuously captures the trunk of the vehicle, generates multiple consecutive second trunk images, and sends a feedback signal (StsOfPhotoShotTrunk = ON) to the entertainment host, and transfers the second trunk images to the entertainment host. The entertainment host analyzes the second trunk images to determine whether there are items in the trunk. If there are items such as a suitcase, the entertainment host immediately sends a first voice prompt message, for example, "Please take all the luggage in the trunk."

[0138] In some embodiments, since the second rear images are captured when the tailgate is in the open state, when a passenger forgets to take the items in the trunk after getting off the vehicle, the second rear images cannot be obtained. At this time, it is possible to determine whether the passenger has taken all the items in the item list according to the item storage signal generated when the passenger stored items in the trunk before getting on the vehicle.

[0139] Specifically, if the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, the entertainment host stores an item storage signal, and the state of the tailgate remains closed from the open state after the passenger gets on the vehicle, it is determined that the passenger has not taken all the items in the item list, and a first voice prompt message is generated, for example, "Please take the luggage in the trunk."

[0140] In this embodiment, by marking the third bounding box of the item to be recognized in the second image group and determining the third movement trajectory, the movement trajectory of the item to be recognized carried by the passenger from inside the vehicle to outside the vehicle can be accurately tracked. Through this movement trajectory, it can be accurately reflected whether the item to be recognized has moved outside the vehicle, and through multiple consecutive images, the recognition result is obtained, rather than only recognizing the item to be recognized through a single image after the passenger gets off the vehicle, reducing the probability of missed recognition and misrecognition of the item to be recognized, thereby improving the recognition accuracy of the item to be recognized.

[0141] In one embodiment, the item anti-loss method further includes:

[0142] 1. Perform item feature recognition on the first in-vehicle image to obtain the first inherent item list of the vehicle.

[0143] Among them, the first in-vehicle image refers to the image obtained by capturing the seat area through the in-vehicle camera when the vehicle arrives at the departure place. The first in-vehicle image applied in this embodiment refers to the image obtained by capturing the seat area through the in-vehicle camera before the passenger gets on the vehicle.

[0144] The first inherent item list refers to the list of items that passengers bring with them or that are fixed on the vehicle before they get on the vehicle, for example, pillows, magazines, etc.

[0145] Specifically, before the vehicle arrives at the departure point and the passengers get on the vehicle, the entertainment host uses computer vision technology, such as a target detection algorithm, to detect and mark the items in the first in-vehicle image by shooting the seat area through the in-vehicle camera, and obtains the location and characteristics of each item. The entertainment host uses an item classification algorithm or model to classify and identify the items based on the location and characteristics of each item. Specifically, it can include identifying the type, color, size and other attributes of the item. Based on the identification results, the entertainment host generates a first inherent item list for the vehicle.

[0146] In some embodiments, the first consumption list of the vehicle can also be obtained by performing feature recognition on the first in-vehicle image. The first consumption list refers to a list of items on the vehicle used to provide consumption services to passengers before the passengers get on the vehicle. The first consumption list is intended to clarify the various consumption services provided by the vehicle to passengers, including but not limited to food, beverages, entertainment facilities, etc. For example, the first consumption list includes bottled water, snacks, etc.

[0147] 2. Perform feature recognition on the second in-vehicle image to obtain a second inherent item list of the vehicle.

[0148] The second in-vehicle image refers to an image obtained by photographing the seat area through the in-vehicle camera when the vehicle arrives at the destination. The second in-vehicle image used in this embodiment refers to an image obtained by photographing the seat area through the in-vehicle camera after the passenger gets off the vehicle.

[0149] The second inherent item list refers to the list of items that passengers bring with them or keep in the vehicle after getting off, for example, pillows, magazines, etc.

[0150] Specifically, after the vehicle arrives at the departure point and the passengers get off, the in-vehicle camera captures the seat area to obtain a second in-vehicle image, and the entertainment host uses computer vision technology, such as a target detection algorithm, to detect and mark the items in the second in-vehicle image and obtain the location and features of each item. The entertainment host uses an item classification algorithm or model to classify and identify the items based on the location and features of each item. Specifically, it may include identifying the type, color, size and other attributes of the item. Based on the identification results, the entertainment host generates a second inherent item list for the vehicle.

[0151] In some embodiments, the second in-vehicle image is subjected to item feature recognition to obtain a second consumption list of the vehicle. The second consumption list refers to a list of items remaining in the vehicle after the passenger gets off the vehicle and used to provide consumption services to the passenger. The second consumption list is intended to clarify the quantity and types of consumer goods remaining in the vehicle after the passenger consumes.

[0152] 3. If the types and / or quantities of items in the second list of inherent items are inconsistent with those in the first list of inherent items, determine the inherent items carried by the passenger in the vehicle, and generate a second voice prompt message to remind the passenger to return the inherent items.

[0153] Among them, the second voice prompt message is a voice prompt message generated for the inherent items carried by the passenger in the vehicle, and is used to remind the passenger to return the items. The second voice prompt message may include information such as the name and location of the items, so that the passenger can understand the items that need to be returned. For example, the second voice prompt message may be "Do not take the items in the vehicle".

[0154] Specifically, the entertainment host compares the generated second list of inherent items with the first list of inherent items to check whether there are inconsistencies in the types and / or quantities of items between the two. If it is found that the second list of inherent items is inconsistent with the first list of inherent items, it is determined that the passenger has carried the inherent items of the vehicle. The entertainment host generates a second voice prompt message to remind the passenger to return the carried inherent items. The entertainment host plays the generated second voice prompt message to the passenger to remind them to pay attention and return the carried inherent items.

[0155] In some embodiments, if the types and / or quantities of items in the second consumption list are inconsistent with those in the first consumption list, it is determined that the passenger has carried consumer goods on the vehicle, and a third voice prompt message is generated to remind the driver that the current passenger carried consumer goods on the vehicle.

[0156] In this embodiment, by performing item feature recognition on the first in-vehicle image and the second in-vehicle image, the first list of inherent items of the inherent items before the passenger gets on the vehicle and the second list of inherent items of the inherent items after the passenger gets off the vehicle can be more accurately recognized, and by comparing the first list of inherent items and the second list of inherent items, it can be determined whether the passenger has carried the inherent items of the vehicle, and whether the types and quantities of these items have changed; if the second list of inherent items is inconsistent with the first list of inherent items, it means that the passenger has carried the inherent items of the vehicle. At this time, generating a second voice prompt message to remind the passenger to return the inherent items can improve the service quality and passenger satisfaction.

[0157] In one embodiment, the item anti-loss method further includes:

[0158] Obtain a payment order; after the payment order is completed, based on the first voice prompt message and the second voice prompt message, generate feedback information, and feedback the feedback information to the mobile device held by the passenger.

[0159] Among them, a payment order refers to a consumption order generated by a passenger during the vehicle ride, including information such as the payment amount, payment method, and payment time.

[0160] Feedback information refers to generating feedback information about the payment order based on the first voice prompt information and the second voice prompt information, including payment results, item return status, etc.

[0161] A mobile device refers to a mobile device such as a mobile phone or tablet held by a passenger, which is used to receive and view feedback information.

[0162] Specifically, the entertainment host obtains the consumption order generated by the passenger during the vehicle ride through the in-vehicle payment system or a third-party payment platform. After the payment order is completed, the entertainment host generates feedback information about the payment order based on the payment result and the passenger's behavior, in combination with the first voice prompt information and the second voice prompt information. The entertainment host uploads the feedback information to the server through TCAM, and the server sends the generated feedback information to the mobile device held by the passenger by means of SMS, APP push, etc., so that the passenger can timely understand information such as the payment result and the item return status.

[0163] In this embodiment, based on the first voice prompt information and the second voice prompt information, feedback information is generated and fed back to the mobile device held by the passenger. In the case where the passenger has left but the item to be identified is still in the vehicle, the passenger is reminded through the feedback information to avoid the problem of the passenger losing the item to be identified in the vehicle.

[0164] In a detailed embodiment, an item anti-loss method includes the following steps:

[0165] 1. When the vehicle arrives at the departure place and the vehicle door is in the open state, the seat area of the vehicle is photographed through the in-vehicle camera to obtain multiple consecutive first in-vehicle images.

[0166] 2. When the tailgate of the vehicle is in the open state, the trunk of the vehicle is photographed through the rear camera to obtain multiple consecutive first trunk images.

[0167] 3. For each image in the first image group, a first bounding box corresponding to the item to be identified carried by the passenger in the image being targeted is marked; the first image group includes multiple consecutive first in-vehicle images and / or first trunk images.

[0168] 4. Based on the positions of the first bounding boxes of the items to be identified in each image in the first image group, the first movement trajectory of the items to be identified is determined.

[0169] 5. When the first movement trajectory indicates that the item to be recognized moves from a preset out-of-vehicle area to a preset in-vehicle area, it is determined that the passenger moves the item to be recognized from outside the vehicle to inside the vehicle, and the type and quantity of the item to be recognized inside the vehicle area are recognized.

[0170] 6. Based on the type and quantity of the item to be recognized inside the vehicle area, an item list of the item to be recognized is determined.

[0171] 7. When the vehicle arrives at the destination, a second image group of the process of the passenger getting off the vehicle with the item to be recognized is obtained through an in-vehicle camera.

[0172] 8. For each image in the second image group, a second bounding box corresponding to the passenger in the targeted image is marked.

[0173] 9. Based on the position of the second bounding box corresponding to the passenger in each image in the second image group, the second movement trajectory of the passenger is determined.

[0174] 10. If the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, it is determined that the passenger has got off the vehicle, and the type and quantity of the item to be recognized in the vehicle area are recognized.

[0175] 11. For each image in the second image group, a third bounding box corresponding to the item to be recognized in the targeted image is marked.

[0176] 12. Based on the position of the third bounding box corresponding to the item to be recognized in each image in the second image group, the third movement trajectory of the item to be recognized is determined.

[0177] 13. If the third movement trajectory indicates that the item to be recognized is in the vehicle area, step 14 is executed; if the third movement trajectory indicates that the item to be recognized is in the out-of-vehicle area, step 15 is executed.

[0178] 14. It is determined that the recognition result indicates that the passenger does not carry all the items in the item list, and step 16 is executed.

[0179] 15. It is determined that the recognition result indicates that the passenger carries all the items in the item list, and a first voice prompt message is generated to remind the passenger to check whether there are any missing items.

[0180] 16. The item features of the first in-vehicle image are recognized to obtain the first inherent item list of the vehicle.

[0181] 17. The item features of the second in-vehicle image are recognized to obtain the second inherent item list of the vehicle.

[0182] XVIII. If the types and / or quantities of items in the second list of inherent items are inconsistent with those in the first list of inherent items, determine the inherent items carried by the passenger in the vehicle and generate a second voice prompt message to remind the passenger to return the inherent items.

[0183] XIX. Obtain a payment order.

[0184] XX. After the payment order is completed, generate a feedback message based on the first voice prompt message and the second voice prompt message, and feedback the feedback message to the mobile device held by the passenger.

[0185] In this embodiment, by obtaining the first image group and the second image group respectively when the vehicle arrives at the departure place and the destination, and immediately performing item feature recognition, an item list of the items to be recognized can be quickly generated when the passenger gets on the vehicle with the items to be recognized, and the recognition result can be quickly generated when the passenger gets off the vehicle. This real-time item recognition and list generation can greatly improve the item management efficiency of the passenger and reduce the possibility of omission or loss. In addition, when the recognition result indicates that the passenger does not carry all the items in the item list, a first voice prompt message is generated to remind the passenger to check whether there are any omitted items and timely remind the passenger to leave the items to be recognized in the vehicle. This real-time reminder can avoid the problem that the reminder fails because the reminder is made only after the passenger has left, and improves the real-time performance and efficiency of the reminder. By recognizing the types and quantities of the items to be recognized through multiple consecutive images instead of only one image, the probability of missed recognition and misrecognition of the items to be recognized is reduced, thereby improving the recognition accuracy of the items to be recognized.

[0186] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are displayed sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0187] Based on the same inventive concept, an item anti-loss device for implementing the above-mentioned item anti-loss method is also provided in an embodiment of the present application. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the item anti-loss device provided below can refer to the limitations on the item anti-loss method in the above text, and will not be repeated here.

[0188] In one embodiment, as Figure 4 shown, an anti-loss device for an item is provided, including:

[0189] A shooting module 401, configured to obtain a first image group of the process of a passenger carrying an item to be identified getting on the vehicle through an in-vehicle camera when the vehicle arrives at the departure place;

[0190] An identification module 402, configured to perform item feature identification on the first image group to obtain an item list of the item to be identified;

[0191] The shooting module 401 is further configured to obtain a second image group of the process of a passenger carrying an item to be identified getting off the vehicle through the in-vehicle camera when the vehicle arrives at the destination;

[0192] The identification module 402 is further configured to perform item feature identification on the second image group to obtain an identification result;

[0193] A reminder module 403, configured to generate a first voice prompt message to remind the passenger to check whether there is any item left if the identification result indicates that the passenger does not carry all the items in the item list.

[0194] In one embodiment, the in-vehicle camera includes an in-vehicle camera and a rear camera. The first image group includes multiple consecutive first in-vehicle images and / or first trunk images. The shooting module 401 is further configured to, when the vehicle arrives at the departure place and the vehicle door is in an open state, shoot the seat area of the vehicle through the in-vehicle camera to obtain multiple consecutive first in-vehicle images; when the rear door of the vehicle is in an open state, shoot the trunk of the vehicle through the rear camera to obtain multiple consecutive first trunk images.

[0195] In one embodiment, the identification module 402 is further configured to, for each image in the first image group, mark a first bounding box corresponding to the item to be identified carried by the passenger in the image targeted; based on the positions of the first bounding boxes of the items to be identified in each image in the first image group, determine the first movement trajectory of the item to be identified; if the first movement trajectory indicates that the item to be identified moves from a preset out-of-vehicle area to a preset in-vehicle area, determine that the passenger moves the item to be identified from outside the vehicle to inside the vehicle, and identify the types and quantities of the items to be identified in the in-vehicle area; based on the types and quantities of the items to be identified in the in-vehicle area, determine the item list of the item to be identified.

[0196] In one embodiment, the recognition module 402 is further configured to, for each image in the second image group, label a second bounding box corresponding to the passenger in the targeted image; determine a second movement trajectory of the passenger based on the positions of the second bounding boxes corresponding to the passengers in each image in the second image group; if the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, determine that the passenger has got off the vehicle, and identify the types and quantities of the items to be recognized in the in-vehicle area; and determine the recognition result based on the types and quantities of the items to be recognized in the in-vehicle area.

[0197] In one embodiment, the recognition module 402 is further configured to, for each image in the second image group, label a third bounding box corresponding to the item to be recognized in the targeted image; determine a third movement trajectory of the item to be recognized based on the positions of the third bounding boxes corresponding to the items to be recognized in each image in the second image group; if the third movement trajectory indicates that the item to be recognized is in the in-vehicle area, determine that the recognition result indicates that the passenger does not carry all the items in the item list.

[0198] In one embodiment, the recognition module 402 is further configured to, if the third movement trajectory indicates that the item to be recognized is in the out-of-vehicle area, determine that the recognition result indicates that the passenger carries all the items in the item list.

[0199] In one embodiment, the recognition module 402 is further configured to perform item feature recognition on the first in-vehicle image to obtain a first list of inherent items of the vehicle; perform item feature recognition on the second in-vehicle image to obtain a second list of inherent items of the vehicle; if the types and / or quantities of the items in the second list of inherent items are inconsistent with those in the first list of inherent items, determine that the passenger carries the inherent items of the vehicle, and generate a second voice prompt message to remind the passenger to return the inherent items.

[0200] In one embodiment, the reminder module 403 is further configured to obtain a payment order; after the payment order is completed, generate a feedback message based on the first voice prompt message and the second voice prompt message, and feedback the feedback message to the mobile device held by the passenger.

[0201] Each module in the above item anti-loss device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0202] In one embodiment, a vehicle is provided, and the vehicle includes a display unit, and the display unit implements the steps of the above item anti-loss method.

[0203] In one embodiment, a computer device is provided, and the computer device can be a vehicle terminal, and its internal structure diagram can be asFigure 5 As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an article anti-loss method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0204] Those skilled in the art can understand that Figure 5 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0205] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0206] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0207] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0208] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0209] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0210] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0211] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for preventing an item from being lost, characterized in that, The method includes: When the vehicle arrives at the departure place, obtaining a first image set of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera; Performing item feature recognition on the first image set to obtain an item list of the item to be recognized; When the vehicle arrives at the destination, obtaining a second image set of the process of the passenger carrying the item to be recognized getting off the vehicle through the in-vehicle camera; Performing item feature recognition on the second image set to obtain a recognition result; If the recognition result indicates that the passenger does not carry all the items in the item list, generating a first voice prompt message to remind the passenger to check whether there are any missing items.

2. The method according to claim 1, wherein The in-vehicle camera includes an in-vehicle camera and a rear camera. The first image set includes multiple consecutive first in-vehicle images and / or first trunk images. When the vehicle arrives at the departure place, obtaining a first image set of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera includes: When the vehicle arrives at the departure place and the vehicle door is in an open state, photographing the seat area of the vehicle through the in-vehicle camera to obtain multiple consecutive first in-vehicle images; When the rear door of the vehicle is in an open state, photographing the trunk of the vehicle through the rear camera to obtain multiple consecutive first trunk images.

3. The method according to claim 1, characterized in that, Performing item feature recognition on the first image set to obtain an item list of the item to be recognized includes: For each image in the first image set, marking a first bounding box corresponding to the item to be recognized carried by the passenger in the image targeted; Based on the positions of the first bounding boxes of the item to be recognized in each image of the first image set, determining a first movement trajectory of the item to be recognized; If the first movement trajectory indicates that the item to be recognized moves from a preset out-of-vehicle area to a preset in-vehicle area, determining that the passenger moves the item to be recognized from outside the vehicle to inside the vehicle, and recognizing the type and quantity of the item to be recognized in the in-vehicle area; Based on the type and quantity of the item to be recognized in the in-vehicle area, determining the item list of the item to be recognized.

4. The method according to claim 1, characterized in that, Performing item feature recognition on the second image set to obtain a recognition result includes: For each image in the second image set, marking a second bounding box corresponding to the passenger in the image targeted; Based on the positions of the second bounding boxes corresponding to the passenger in each image of the second image set, determining a second movement trajectory of the passenger; If the second movement trajectory indicates that the passenger moves from a preset in-vehicle area to a preset out-of-vehicle area, determining that the passenger has got off the vehicle, and recognizing the type and quantity of the item to be recognized in the in-vehicle area; Based on the type and quantity of the item to be recognized in the in-vehicle area, determining the recognition result.

5. The method according to claim 4, wherein Based on the type and quantity of the item to be recognized in the in-vehicle area, determining the recognition result includes: For each image in the second image set, marking a third bounding box corresponding to the item to be recognized in the image targeted; Based on the positions of the third bounding boxes corresponding to the item to be recognized in each image of the second image set, determining a third movement trajectory of the item to be recognized; If the third motion trajectory indicates that the item to be recognized is in the in-vehicle area, it is determined that the recognition result indicates that the passenger does not carry all the items in the item list.

6. The method according to claim 5, characterized in that, The method further includes: If the third motion trajectory indicates that the item to be recognized is in the out-of-vehicle area, it is determined that the recognition result indicates that the passenger carries all the items in the item list.

7. The method according to claim 1, wherein The first image group includes first in-vehicle images, and the second image group includes second in-vehicle images. The method further includes: Performing item feature recognition on the first in-vehicle images to obtain a first list of inherent items of the vehicle; Performing item feature recognition on the second in-vehicle images to obtain a second list of inherent items of the vehicle; If the types and / or quantities of items in the second list of inherent items are inconsistent with those in the first list of inherent items, it is determined that the passenger carries the inherent items of the vehicle, and a second voice prompt message is generated to remind the passenger to return the inherent items.

8. The method according to claim 7, wherein The method further includes: Obtaining a payment order; After the payment order is completed, based on the first voice prompt message and the second voice prompt message, generating a feedback message and feeding back the feedback message to the mobile device held by the passenger.

9. An anti-loss device for an item, characterized in that, The device includes: A photographing module, configured to obtain a first image group of the process of a passenger carrying an item to be recognized getting on the vehicle through an in-vehicle camera when the vehicle arrives at the departure place; An identification module, configured to perform item feature recognition on the first image group to obtain an item list of the item to be recognized; The photographing module is further configured to obtain a second image group of the process of the passenger carrying the item to be recognized getting off the vehicle through the in-vehicle camera when the vehicle arrives at the destination; The identification module is further configured to perform item feature recognition on the second image group to obtain a recognition result; A reminder module, configured to generate a first voice prompt message to remind the passenger to check whether there is any item omission if the recognition result indicates that the passenger does not carry all the items in the item list.

10. A vehicle, characterized in that, The vehicle includes a display unit, and the display unit implements the steps of the method according to any one of claims 1 to 8.

11. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

Images