Method, apparatus, computer device, and storage medium for handling legacy objects

Abstract

The present application relates to a method, apparatus, and computer device for handling left-behind objects. The method includes: obtaining an image to be detected, and inputting the image to be detected into a left-behind object detection model to perform left-behind object detection, so as to obtain the existence probability of left-behind objects corresponding to the image to be detected; determining that there are left-behind objects in the image to be detected based on the existence probability of left-behind objects; sending a left-behind object retrieval prompt to a target client, where the left-behind object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position. Using this method can improve the processing efficiency of left-behind objects.

Description

Technical Field

[0001] This application relates to the field of automotive safety technologies, and particularly to a method, apparatus, computer device, storage medium, and computer program product for handling left-behind objects. Background Art

[0002] With the development of automotive technologies, the emergence of intelligent vehicles has made people's travel more convenient. People can travel by self-driving or by calling a taxi. However, after passengers get off the vehicle, items left behind by passengers often appear in the vehicle cockpit, such as mobile phones, computers, wallets, etc. If passengers cannot retrieve the left-behind items in time, they will suffer losses. When living beings such as young children or pets are forgotten in the vehicle, there will also be life hazards. Therefore, it is necessary to handle left-behind objects in a timely manner. The existing method for handling left-behind objects is to collect images before and after passengers get on the vehicle, and judge whether there are left-behind objects in the vehicle by the difference obtained by subtracting the images. Then, the passengers are contacted to pick up the left-behind objects. However, when the image difference is small, it is not possible to accurately judge whether there are left-behind objects in the vehicle by the image difference, and there may even be misjudgments of left-behind objects. Therefore, it takes a longer time to judge the image difference, resulting in the inability to return the left-behind objects to the passengers in time, thus causing the problem of low efficiency in handling left-behind objects. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for handling left-behind objects that can quickly identify left-behind objects, handle left-behind objects in a timely manner, and improve the efficiency of handling left-behind objects.

[0004] In a first aspect, this application provides a method for handling left-behind objects. The method includes:

[0005] Obtain a to-be-detected image, and input the to-be-detected image into a left-behind object detection model for left-behind object detection to obtain the probability of the existence of a left-behind object corresponding to the to-be-detected image;

[0006] Determine that there is a left-behind object in the to-be-detected image based on the probability of the existence of a left-behind object;

[0007] Send a left-behind object retrieval prompt to a target client, where the left-behind object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position.

[0008] In one embodiment, determining that there is a left-behind object in the to-be-detected image based on the probability of the existence of a left-behind object includes:

[0009] When it is detected that the probability of the existence of a left-behind object meets a preset left-behind object category recognition condition, determine that there is a left-behind object in the to-be-detected image; or

[0010] When it is detected that the probability of the existence of the remaining object meets the preset re-inspection condition, a re-inspection request for the remaining object is sent to the control end. The re-inspection request for the remaining object carries the image to be detected, and the re-inspection result of the remaining object returned by the control end is obtained. Based on the re-inspection result of the remaining object, it is determined that there is a remaining object in the image to be detected.

[0011] In one embodiment, after it is determined that there is a remaining object in the image to be detected when it is detected that the probability of the existence of the remaining object meets the preset remaining object category recognition condition, the following steps are further included:

[0012] The image to be detected is input into the remaining object classification model for classification and recognition to obtain the degree of the remaining object target living body category corresponding to the image to be detected;

[0013] When the degree of the remaining object target living body category meets the preset vehicle control condition, a safety control instruction is sent to the vehicle end so that the vehicle end performs safety control on the corresponding vehicle based on the safety control instruction;

[0014] The safety control result returned by the vehicle end is obtained, and a re-inspection request for the remaining object living body is sent to the control end. The re-inspection request for the remaining object living body carries the image to be detected;

[0015] The target living body category confirmation result corresponding to the image to be detected returned by the control end is obtained.

[0016] In one embodiment, after the target living body category confirmation result corresponding to the image to be detected returned by the control end is obtained, the following steps are included:

[0017] The image to be detected is used as a training sample, and the remaining object existence label and the target living body category label are determined based on the target living body category confirmation result;

[0018] The remaining object detection model is updated and trained using the training sample and the remaining object existence label to obtain an updated remaining object detection model;

[0019] The remaining object classification model is updated and trained using the training sample and the target living body category label to obtain an updated remaining object classification model.

[0020] In one embodiment, after a remaining object retrieval prompt is sent to the target client, the remaining object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position, the following steps are further included:

[0021] The target extraction position and the target extraction time returned by the target client are obtained. The target extraction position is selected from the set of candidate positions, and the target extraction time is selected from the retrieval times corresponding to each candidate position;

[0022] Obtain the current vehicle position corresponding to the vehicle end, and determine the current driving distance based on the current vehicle position and the target extraction position;

[0023] Calculate the driving speed corresponding to the vehicle end based on the current driving distance and the target extraction time;

[0024] Send a driving control instruction to the vehicle end, where the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle end travels at the driving speed and reaches the target extraction position at the target extraction time.

[0025] In one embodiment, the method further includes:

[0026] Obtain the arrival time of the vehicle end at the target extraction position, and calculate the waiting time corresponding to the vehicle end based on the arrival time and the target extraction time;

[0027] Obtain the available stay time corresponding to the target extraction position. When it is detected that the waiting time is greater than the available stay time, generate waiting route information based on the waiting time and the target extraction position;

[0028] Send the waiting route plan to the vehicle end, so that the vehicle end controls the corresponding vehicle to travel along the waiting route and reaches the target extraction position at the target extraction time.

[0029] In one embodiment, after sending a control instruction to the vehicle end based on the target extraction position and the target extraction time, so that the vehicle end controls the corresponding vehicle to reach the target extraction position at the target extraction time, it further includes:

[0030] When no identity recognition request sent by the vehicle end is detected after a preset time period, obtain the service point position;

[0031] Send a driving instruction to the vehicle end based on the service point position, so that the vehicle end travels to the service point;

[0032] Generate legacy object position information based on the service point position, and send the legacy object position information to the target client.

[0033] In a second aspect, the present application further provides a legacy object processing device. The device includes:

[0034] A detection module, configured to obtain an image to be detected, and input the image to be detected into a legacy object detection model for legacy object detection, to obtain the probability of the existence of a legacy object corresponding to the image to be detected;

[0035] An identification module, configured to determine that there is a legacy object in the image to be detected based on the probability of the existence of the legacy object;

[0036] A reminder module for sending a reminder for retrieving a legacy object to a target client, where the reminder for retrieving a legacy object includes a set of candidate locations and the retrieval time corresponding to each candidate location.

[0037] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Obtain a to-be-detected image, input the to-be-detected image into a legacy object detection model for legacy object detection, and obtain the probability of the existence of a legacy object corresponding to the to-be-detected image;

[0039] Determine that there is a legacy object in the to-be-detected image based on the probability of the existence of a legacy object;

[0040] Send a reminder for retrieving a legacy object to a target client, where the reminder for retrieving a legacy object includes a set of candidate locations and the retrieval time corresponding to each candidate location.

[0041] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0042] Obtain a to-be-detected image, input the to-be-detected image into a legacy object detection model for legacy object detection, and obtain the probability of the existence of a legacy object corresponding to the to-be-detected image;

[0043] Determine that there is a legacy object in the to-be-detected image based on the probability of the existence of a legacy object;

[0044] Send a reminder for retrieving a legacy object to a target client, where the reminder for retrieving a legacy object includes a set of candidate locations and the retrieval time corresponding to each candidate location.

[0045] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0046] Obtain a to-be-detected image, input the to-be-detected image into a legacy object detection model for legacy object detection, and obtain the probability of the existence of a legacy object corresponding to the to-be-detected image;

[0047] Determine that there is a legacy object in the to-be-detected image based on the probability of the existence of a legacy object;

[0048] Send a reminder for retrieving a legacy object to a target client, where the reminder for retrieving a legacy object includes a set of candidate locations and the retrieval time corresponding to each candidate location.

[0049] The above-mentioned method, apparatus, computer device, storage medium and computer program product for handling left-behind objects detect left-behind objects in the image to be detected by using a left-behind object detection model, and obtain the probability of the existence of left-behind objects corresponding to the image to be detected. When the server determines that there are left-behind objects in the image to be detected according to the probability of the existence of left-behind objects, it sends a left-behind object retrieval prompt to the target client, which can timely prompt the customer that they have left behind objects. And the left-behind object retrieval prompt includes a set of candidate locations and the retrieval time corresponding to each candidate location, which can timely provide the customer with optional left-behind object extraction locations and extraction times, avoid the long waiting of the customer, and can timely return the left-behind objects to the target customer, thereby improving the processing efficiency of left-behind objects. Brief Description of the Drawings

[0050] Figure 1 It is an application environment diagram of the left-behind object handling method in an embodiment;

[0051] Figure 2 It is a schematic flowchart of the left-behind object handling method in an embodiment;

[0052] Figure 3 It is a schematic flowchart of the left-behind object classification and recognition in an embodiment;

[0053] Figure 4 It is a schematic flowchart of calculating the driving speed in an embodiment;

[0054] Figure 5 It is a schematic flowchart of generating a waiting route in an embodiment;

[0055] Figure 6 It is a schematic flowchart of the left-behind object handling in a specific embodiment;

[0056] Figure 7 It is a structural block diagram of the left-behind object handling apparatus in an embodiment;

[0057] Figure 8 It is an internal structure diagram of a computer device in an embodiment;

[0058] Figure 9 It is an internal structure diagram of a computer device in another embodiment. Detailed Description of the Embodiments

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The left-behind object handling method provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the vehicle terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. The server 104 obtains the image to be detected uploaded by the vehicle terminal 102, and inputs the image to be detected into the legacy object detection model for legacy object detection, obtaining the probability of the existence of the legacy object corresponding to the image to be detected. The server 104 can also obtain the probability of the existence of the legacy object corresponding to the image to be detected through the vehicle terminal 102. The probability of the existence of the legacy object corresponding to the image to be detected can be obtained after the vehicle terminal performs legacy object detection on the image to be detected and uploads the probability of the existence of the legacy object to the server 104; when the server 104 detects that the probability of the existence of the legacy object meets the preset re-inspection condition, it sends a legacy object re-inspection request to the control terminal, and the legacy object re-inspection request carries the image to be detected; the server 104 obtains the legacy object re-inspection result returned by the control terminal. When the legacy object re-inspection result is that there is a legacy object, it obtains each candidate extraction position and each candidate extraction time; the server 104 sends each candidate extraction position and each candidate extraction time to the target client associated with the vehicle terminal, and obtains the target extraction position and the target extraction time returned by the target client. The target extraction position is selected from each candidate extraction position, and the target extraction time is selected from each candidate extraction time; the server 104 sends a control instruction to the vehicle terminal based on the target extraction position and the target extraction time, so that the vehicle terminal controls the corresponding vehicle to drive to the target extraction position at the target extraction time. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0061] In one embodiment, as Figure 2 shown, a method for processing legacy objects is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0062] Step 202, obtain the image to be detected, and input the image to be detected into the legacy object detection model for legacy object detection, obtaining the probability of the existence of the legacy object corresponding to the image to be detected.

[0063] Among them, the image to be detected refers to the image corresponding to the target area in the vehicle at the vehicle end. The vehicle-mounted end refers to the terminal installed on the vehicle that can communicate with the server. The vehicle can be an autonomous vehicle or a vehicle controlled manually. The target area refers to the area in the vehicle where there may be left objects. For example, the vehicle cockpit area. The left object detection model refers to a trained model used to detect whether there are left objects in the image to be detected. The probability of the existence of a left object refers to the detection result output by the left object detection model after detecting the left object in the image to be detected, which is used to characterize the possibility of the existence of a left object in the image to be detected. The higher the probability of existence, the higher the possibility of the existence of a left object in the vehicle.

[0064] Specifically, the image to be detected can be the image corresponding to the target area in the vehicle collected by the vehicle end through a camera device, and then the vehicle end uploads the image to be detected to the server. After receiving the image to be detected uploaded by the vehicle end, the server calls the left object detection model from the model storage system. The left object detection model can be pre-stored in the model storage system and is a trained artificial intelligence model. The server inputs the image to be detected into the left object detection model to calculate the possibility of the existence of a left object, and then the server obtains the probability of the existence of a left object corresponding to the image to be detected output by the left object detection model. The vehicle end can also call the left object detection model from the local storage space after collecting the image to be detected, input the image to be detected into the left object detection model to detect the left object, obtain the probability of the existence of a left object corresponding to the image to be detected, and then upload the probability of the existence of a left object corresponding to the image to be detected to the server. The server can directly obtain the probability of the existence of a left object corresponding to the image to be detected through the vehicle end.

[0065] Step 204: Determine that there is a left object in the image to be detected based on the probability of the existence of the left object.

[0066] Specifically, after obtaining the probability of the existence of the left object, the server can use the pre-set judgment condition for determining whether there is a left object in the image to be detected to judge the probability of the existence of the left object, and determine that there is a left object in the image to be detected according to the judgment result.

[0067] Step 206: Send a reminder to retrieve the left object to the target client. The reminder to retrieve the left object includes a set of candidate positions and the retrieval time corresponding to each candidate position.

[0068] Among them, the target client refers to the user terminal associated with the vehicle terminal. The user associates the identity information with the vehicle terminal through this user terminal, so that the user can use the vehicle corresponding to the vehicle terminal. The target customer refers to the user object to which the legacy object belongs. The legacy object retrieval prompt refers to the prompt information used to prompt the target customer that the legacy object has been left. The candidate location set refers to the set of locations where the vehicle corresponding to the vehicle terminal can stay that are pre-set by the server on the map, which can be the set of locations where the legacy object can be retrieved. The retrieval time refers to the time pre-set corresponding to the candidate location and available for the target customer to select for retrieving the legacy object.

[0069] Specifically, when the server determines that there is a legacy object in the image to be detected, it determines that there is a legacy object in the target area of the vehicle corresponding to the vehicle terminal. Then the server can obtain the current location of the vehicle corresponding to the vehicle terminal, and obtain each pre-set stoppable location on the map according to the current location of the vehicle, and use the obtained stoppable locations as the candidate location set. Then the server obtains the current time, and determines the retrieval time corresponding to each candidate location within the maximum retrievable time according to the current time. The maximum retrievable time can be the longest time for providing the retrieval service. Then the server can generate a legacy object retrieval prompt according to the candidate location set, the retrieval time corresponding to each candidate location, and the image to be detected, so that the target customer can understand the situation of the legacy object left and the retrieval information of the relevant legacy object through the legacy object retrieval prompt.

[0070] In the above legacy object processing method, by using the legacy object detection model to detect the legacy object in the image to be detected, the probability of the existence of the legacy object corresponding to the image to be detected is obtained. When the server determines that there is a legacy object in the image to be detected according to the probability of the existence of the legacy object, it sends a legacy object retrieval prompt to the target client, which can prompt the customer in time that the legacy object has not been taken away. And the legacy object retrieval prompt includes the candidate location set and the retrieval time corresponding to each candidate location, which can provide the customer with the selectable legacy object retrieval location and retrieval time in time, avoid the long waiting of the customer, and return the legacy object to the target customer in time, thereby improving the processing efficiency of the legacy object.

[0071] In one embodiment, step 204, determining that there is a legacy object in the image to be detected based on the probability of the existence of the legacy object, includes:

[0072] When it is detected that the probability of the existence of the legacy object meets the preset legacy object category recognition condition, it is determined that there is a legacy object in the image to be detected; or

[0073] When it is detected that the probability of the existence of a left-behind object meets the preset re-inspection condition, a re-inspection request for the left-behind object is sent to the control end. The re-inspection request for the left-behind object carries the image to be detected, and the re-inspection result of the left-behind object returned by the control end is obtained. Based on the re-inspection result of the left-behind object, it is determined that there is a left-behind object in the image to be detected.

[0074] Among them, the preset recognition condition for the category of left-behind objects refers to the judgment condition set in advance for determining whether the image to be detected needs to be recognized for the category of left-behind objects. The preset re-inspection condition refers to the judgment condition set in advance for determining whether the image to be detected needs to be re-inspected. The control end refers to the control platform used to monitor and control the safe operation of the corresponding vehicle at the vehicle end.

[0075] Specifically, the server can use two methods to determine that there is a left-behind object in the image to be detected. When the server detects that the probability of the existence of a left-behind object meets the preset recognition condition for the category of left-behind objects, that is, when the probability of the existence of a left-behind object in the image to be detected reaches the preset threshold for confirming the existence of a left-behind object, the server determines that there is a left-behind object in the image to be detected and meets the preset recognition condition for the category of left-behind objects. Or when the server detects that the probability of the existence of a left-behind object is within the preset re-inspection range, it meets the preset re-inspection condition. The preset re-inspection range refers to the range of the probability of the existence of a left-behind object set in advance. For example, the re-inspection range can be the middle range of the existence probability. Then, a re-inspection request for the left-behind object is generated according to the image to be detected and sent to the control end. The control end reinspects whether there is a left-behind object in the image to be detected according to the image to be detected in the re-inspection request for the left-behind object.

[0076] In this embodiment, when the probability of the existence of a left-behind object reaches the preset threshold for confirming the existence of a left-behind object, the server can quickly determine that there is a left-behind object in the image to be detected. Or when the probability of the existence of a left-behind object is at an intermediate level, it meets the preset re-inspection condition and sends a re-inspection request for the left-behind object to the control end. By reinspecting the image to be detected through the control end, it can quickly and accurately judge whether there is a left-behind object in the image to be detected. Therefore, when the probability of the existence of a left-behind object meets different conditions, the server can quickly determine whether there is a left-behind object in the image to be detected, so as to process the left-behind object in time and improve the processing efficiency of the left-behind object.

[0077] In one embodiment, as Figure 3 shown, a flow schematic diagram for classifying and recognizing left-behind objects is provided. After it is determined that there is a left-behind object in the image to be detected when the probability of the existence of a left-behind object is detected to meet the preset recognition condition for the category of left-behind objects, it further includes:

[0078] Step 302, input the image to be detected into the left-behind object classification model for classification and recognition to obtain the degree of the living body category of the left-behind object target corresponding to the image to be detected.

[0079] Step 304: When the degree of the target living category of the left-behind object meets the preset vehicle control condition, send a safety control instruction to the vehicle side, so that the vehicle side performs safety control on the corresponding vehicle based on the safety control instruction.

[0080] Step 306: Obtain the safety control result returned by the vehicle side, and send a left-behind object living body re-inspection request to the control side. The left-behind object living body re-inspection request carries the image to be detected.

[0081] Step 308: Obtain the confirmation result of the target living category corresponding to the image to be detected returned by the control side.

[0082] Among them, the left-behind object classification model refers to a trained model used to classify and identify the left-behind objects in the image to be detected. The degree of the target living category of the left-behind object refers to the probability that the left-behind object belongs to the living category in each category, and is used to represent whether the left-behind object in the image to be detected belongs to the living category. The preset vehicle control condition refers to the judgment condition for determining whether to control the vehicle environment based on the degree of the target living category of the left-behind object. The safety control instruction is an instruction for controlling the vehicle environment according to the environment required for the living body to maintain a normal life state. The safety control result refers to the in-vehicle environment information returned by the vehicle side after performing safety control. The confirmation result of the target living category refers to the result of the control side confirming whether the left-behind object in the image to be detected belongs to the living category through re-inspection.

[0083] Specifically, after the server determines that the image to be detected meets the preset left-behind object category recognition condition, the server inputs the image to be detected into the left-behind object classification model for classification and recognition. The left-behind object classification model performs classification calculation on the left-behind object in the image to be detected according to the pre-trained model parameters, and obtains the category probability of the left-behind object in the image to be detected. According to the category probability of the left-behind object, when the category probability is the probability of belonging to the living category, the category probability is used as the degree of the target living category of the left-behind object. When the degree of the target living category of the left-behind object exceeds the preset category threshold, it indicates that the left-behind object is of the living category. At this time, the existence probability of the left-behind object meets the preset vehicle control condition.

[0084] Then the server sends a safety control instruction to the vehicle end, so that the vehicle end performs safety control on the corresponding vehicle of the vehicle end when receiving the safety control instruction. After the vehicle end detects that the safety control is completed, the vehicle end collects the vehicle environment information, generates the safety control result according to the vehicle environment information and uploads the safety control result to the server. In a specific embodiment, the server can send a liveness detection instruction to the vehicle end when it detects that the probability of the existence of the legacy object meets the preset legacy object category recognition condition. So that when the vehicle end receives the liveness detection instruction, the vital signs sensor is used to detect the vital signs of the target area in the vehicle. When the vehicle end detects the existence of vital signs, the liveness prompt information is generated and uploaded to the server, and the vehicle end performs safety control on the vehicle. The server performs corresponding processing after receiving the liveness prompt information. In a specific embodiment, when the vehicle end receives the safety control instruction, it obtains the in-vehicle environment information, such as the temperature, humidity, oxygen content, etc. in the vehicle. When the vehicle end detects that the vehicle environment does not meet the environment required for the normal vital signs of the living body, the vehicle equipment is controlled in real time according to the in-vehicle environment, such as opening the window, turning on the ventilation, turning on the air conditioner, etc., so that the in-vehicle environment meets the environment required for the living body to maintain a normal life state. After receiving the safety control result returned by the vehicle end, the server sends a live recheck request for the remaining object to the control end, the live recheck request for the remaining object carries the image to be detected, and receives the target live category confirmation result returned by the control end after rechecking whether the remaining object in the image to be detected belongs to the live category. In a specific embodiment, the control end can manually recheck the image to be detected. When the recheck result shows that there is a live body in the image to be detected, the control end initiates a communication request to the target client so that the target client can understand the live status through the target client.

[0085] In a specific embodiment, when the server detects that the probability of the existence of a legacy object in the image to be detected reaches a preset re-inspection range, for example, the re-inspection range is an intermediate probability range of 45%-65%, the preset re-inspection condition is met at this time, and the server sends a legacy object re-inspection request to the control end, so that the control end re-inspects the image to be detected. When the existence probability is greater than 65%, it is considered that a legacy object exists, and when the existence probability is less than 45%, it is considered that no legacy object exists.

[0086] In this embodiment, by classifying and identifying the image to be detected after detecting the presence of the leftover object in the image to be detected, it is possible to timely determine whether the leftover object in the image to be detected belongs to the living body category. When it is detected that the leftover object belongs to the living body category, a safety control instruction is promptly sent to the vehicle end, so that the vehicle end can quickly control the in-vehicle environment, thereby ensuring the safety of the living body in the vehicle.

[0087] In one embodiment, step 308, after obtaining the target living body category confirmation result corresponding to the image to be detected returned by the control end, further includes:

[0088] Use the image to be detected as a training sample, and determine the presence label of the left-behind object and the target live object category label based on the confirmation result of the target live object category;

[0089] Use the training sample and the presence label of the left-behind object to update and train the left-behind object detection model to obtain an updated left-behind object detection model;

[0090] Use the training sample and the target live object category label to update and train the left-behind object classification model to obtain an updated left-behind object classification model.

[0091] Among them, the presence label of the left-behind object refers to the image label corresponding to the image to be detected used for updating and training the left-behind object detection model. The target live object category label refers to the image label corresponding to the image to be detected used for updating and training the left-behind object classification model. The updated left-behind object detection model and the updated left-behind object classification model are obtained by respectively updating the model parameters in the left-behind object detection model and the left-behind object classification model.

[0092] Specifically, the server uses the detected image as a training sample, and respectively generates the presence label of the left-behind object and the target live object category label according to the confirmation result of the target live object category. The server associates the presence label of the left-behind object and the target live object category label with the training sample respectively. Then the server respectively puts the left-behind object detection model and the left-behind object classification model into the model update training state. The server inputs the training sample and the presence label of the left-behind object into the left-behind object detection model to update the model parameters, and obtains an updated left-behind object detection model. The server inputs the training sample and the target live object category label into the left-behind object classification model to update and train the model parameters, and obtains an updated left-behind object classification model. Then the server stores the updated left-behind object detection model and the updated left-behind object classification model in the model storage space for direct calling when used later.

[0093] In this embodiment, by using the image to be redetected as a training sample, and then inputting the training sample into the left-behind object detection model and the left-behind object classification model respectively for update training, the detection accuracy of the left-behind object in the image to be detected by the obtained updated left-behind object detection model is higher, and the recognition accuracy of the left-behind object in the image to be detected by the obtained updated left-behind object classification model is higher. Thus, the processing efficiency of the left-behind object is improved.

[0094] In one embodiment, as Figure 4 shown, a flow diagram for calculating the driving speed is provided; Step 206, after sending a left-behind object retrieval prompt to the target client, where the left-behind object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position, further includes:

[0095] Step 402: Obtain the target extraction location and target extraction time returned by the target client. The target extraction location is selected from the candidate location set, and the target extraction time is selected from the retrieval times corresponding to each candidate location.

[0096] Step 404: Obtain the current vehicle location corresponding to the vehicle terminal, and determine the current driving distance based on the current vehicle location and the target extraction location.

[0097] Step 406: Calculate the driving speed corresponding to the vehicle terminal based on the current driving distance and the target extraction time.

[0098] Step 408: Send a driving control instruction to the vehicle terminal. The driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle terminal travels at the driving speed and reaches the target extraction location at the target extraction time.

[0099] Among them, the target extraction location and target extraction time refer to the extraction location and extraction time selected by the target customer from the candidate location set and the retrieval times corresponding to each candidate location in the target client. The current vehicle location refers to the location where the vehicle stays at the current time. The current driving distance refers to the distance that the vehicle travels from the current vehicle location to the target extraction location. The driving speed refers to the driving speed of the vehicle during the driving process from the current vehicle location to the target extraction location. The driving control instruction refers to the instruction for controlling the driving speed of the vehicle.

[0100] Specifically, the server sends a reminder for retrieving the left-behind object to the target client associated with the vehicle terminal, and obtains the target extraction location and target extraction time returned by the target client. For example, the server sends a reminder for retrieving the left-behind object to be displayed on the target client, so that the target customer selects the target extraction location from the candidate locations displayed on the target client and selects the target extraction time from the retrieval times corresponding to each candidate location, and then the server receives the target extraction location and target extraction time sent by the target client. The reminder for retrieving the left-behind object carries the image to be detected, and the image to be detected carried can be an image with sensitive information hidden. For example, the server uses mosaics to blur sensitive information such as pedestrians outside the window and store information on the image to be detected.

[0101] After receiving the target extraction time and target extraction location returned by the target client, the server obtains the current vehicle location corresponding to the vehicle end. The current vehicle location can be the passenger getting-off location. The server can obtain the route between the current vehicle location and each target extraction location on the map. The server determines the current driving distance in the route between the current vehicle location and each target extraction location according to the target extraction time, and calculates the driving speed corresponding to the vehicle end according to the target extraction time and the current driving distance. In a specific embodiment, when the target extraction time is short, a shorter route is selected as the current driving distance in the route between the current vehicle location and each target extraction location, and the driving speed when the vehicle at the vehicle end drives the shorter route is calculated according to the current driving distance and the target extraction time; when the target extraction time is long, a longer route is selected as the current driving distance in the route between the current vehicle location and each target extraction location, and the driving speed when the vehicle at the vehicle end drives the longer route is calculated according to the current driving distance and the target extraction time.

[0102] The server sends a driving control instruction to the vehicle end. The driving control instruction carries the driving speed and the driving route, so that the vehicle corresponding to the vehicle end drives according to the driving speed and the driving route, and drives to the target extraction location at the target extraction time.

[0103] In this embodiment, the current driving distance is determined by the current vehicle location and the target extraction location, and then the driving speed corresponding to the vehicle end is calculated using the target extraction time and the current driving distance, which can enable the vehicle to drive to the target extraction location on time at the target extraction time, enabling the passenger to extract the left-behind object in time, thereby improving the processing efficiency of the left-behind object.

[0104] In one embodiment, as Figure 5 shown, a flowchart of generating a waiting route is provided; the method further includes:

[0105] Step 502, obtain the arrival time of the vehicle end at the target extraction location, and calculate the waiting time corresponding to the vehicle end based on the arrival time and the target extraction time;

[0106] Step 504, obtain the available stay time corresponding to the target extraction location. When it is detected that the waiting time is greater than the available stay time, generate waiting route information based on the waiting time and the target extraction location;

[0107] Step 506, send the waiting route plan to the vehicle end, so that the vehicle end controls the corresponding vehicle to drive according to the waiting route and drives to the target extraction location at the target extraction time.

[0108] Among them, the waiting time refers to the time that the vehicle needs to wait for the extraction of the left-behind object when it arrives at the target extraction position. The allowable stay time refers to the time that the vehicle is allowed to stay as specified. The waiting route refers to the route used to consume the waiting time when the waiting time of the vehicle exceeds the allowable stay time.

[0109] Specifically, the server obtains the arrival time of the vehicle terminal at the target extraction position, calculates the difference between the arrival time and the target extraction time, and obtains the corresponding waiting time of the vehicle terminal. The server obtains the allowable stay time corresponding to the target extraction position, and compares the allowable stay time with the waiting time. When it is detected that the waiting time is greater than the allowable stay time, the server generates waiting route information in the map according to the waiting time. The driving time of the waiting route can be equal to the waiting time or less than the waiting time, and the difference between the driving time and the waiting time does not exceed the allowable stay time. For example, if the vehicle's waiting time is 10 minutes and the allowable stay time is 5 minutes, the driving time of the waiting route can be 10 minutes or 6 - 9 minutes. Then the server sends the waiting route information to the vehicle terminal, so that the vehicle terminal controls the corresponding vehicle to drive along the waiting route and arrive at the target extraction position at the target extraction time. The server can also search for each parking point within the preset range of the target extraction position in the map according to the target extraction position, plan the parking point route to each parking point according to the allowable stay time and waiting time of each parking point, and send the parking point route to the vehicle terminal, so that after the time at the target parking position reaches the allowable stay time, the vehicle terminal controls the corresponding vehicle to drive to other parking points to stay and arrive at the target extraction position at the target extraction time.

[0110] In this embodiment, by generating the waiting route of the vehicle, when the vehicle arrives at the target extraction position in advance, it can return to the target extraction position at the target extraction time by driving along the waiting route without violating the allowable stay time of traffic rules, and wait for the passenger to extract the left-behind object. Thus, the left-behind object can be returned to the passenger in time, improving the passenger experience and the processing efficiency of the left-behind object.

[0111] In one embodiment, after step 210 of sending a control instruction to the vehicle terminal based on the target extraction position and the target extraction time, so that the vehicle terminal controls the corresponding vehicle to drive to the target extraction position at the target extraction time, it further includes:

[0112] When the identity recognition request sent by the vehicle terminal is not detected after a preset time period, obtain the service point position;

[0113] Send a driving instruction to the vehicle terminal based on the service point position, so that the vehicle terminal drives to the service point;

[0114] Generate left-behind object position information based on the service point position, and send the left-behind object position information to the target client.

[0115] Among them, the identity recognition request refers to the identity confirmation request sent by the vehicle end when the target customer conducts identity recognition on the vehicle end. The service point location refers to the location of the service area that provides services and can store the left-behind object.

[0116] Specifically, after the server detects that the vehicle corresponding to the vehicle end travels to the target extraction position at the target extraction time, it starts timing. When the server does not receive the identity recognition request sent by the vehicle end after detecting the expected time period, it determines that the target customer does not extract the left-behind object within the expected time period, and obtains the service point location, which can be the service point location closest to the target extraction position. Then the server generates a driving instruction according to the service point location and sends the driving instruction to the vehicle end, so that the vehicle end travels to the service point according to the service point location and places the left-behind object at the service point for storage. Then the server generates the left-behind object location information according to the service point location and sends the left-behind object location information to the target client, so that the object can know the location of the left-behind object through the target client. In this embodiment, when it is detected that the passenger is overdue, sending the left-behind object to the service point can properly place the left-behind object, thereby ensuring the property safety of the passenger.

[0117] In a specific embodiment, as Figure 6 shown, a schematic flowchart of the processing of the left-behind object is provided;

[0118] When the vehicle terminal determines that a passenger has left the cockpit when no passenger is detected in the cockpit, the vehicle terminal then collects cockpit images through a camera device and sends the cockpit images to the server. After receiving the cockpit images, the server uses a classification algorithm, such as a lightweight deep neural network algorithm, to perform classification calculations on the cockpit images. The classification categories include the category of left-behind objects and the category of no left-behind objects, and obtains the probability that there are left-behind objects in the cockpit images. The server can also use a traditional algorithm, such as OpenCV (Open Source Computer Vision Library), to compare the cockpit images after the passengers get off with the images of no left-behind objects, obtain the area of the left-behind objects, and then calculate the ratio of the area of the left-behind objects to the cockpit images to obtain the probability that there are left-behind objects in the cockpit images. Then the server compares the probability that there are left-behind objects in the cockpit images with a threshold A. The threshold A represents a preset threshold for confirming the existence of left-behind objects. When the probability of the existence of left-behind objects is greater than the threshold A, it is determined that there are left-behind objects in the cockpit images. A left-behind object retrieval prompt is generated and sent to the user's mobile terminal. The prompt information includes the cockpit image with sensitive information already hidden, and a suspension operation instruction is sent to the vehicle terminal, so that the vehicle terminal pauses receiving passenger orders after receiving the stop operation instruction. When the passenger returns to the vehicle within the allowable stay time stipulated by the traffic rules corresponding to getting off, the server identifies the passenger through the vehicle terminal. After successful identification, the door is opened for the passenger to pick up the left-behind object. Then the server performs left-behind object detection on the cockpit images collected again by the vehicle terminal. When it is detected that there are no left-behind objects in the cockpit, an operation instruction is sent to the vehicle terminal to enable the corresponding vehicle of the vehicle terminal to continue operating. When the probability of the existence of left-behind objects is less than the threshold A, it means that there are no left-behind objects in the cockpit. The server sends an operation instruction to the vehicle terminal to enable the corresponding vehicle of the vehicle terminal to continue operating.

[0119] After the server confirms that there are left-behind objects in the cockpit images, it uses an object detection algorithm, such as SSD, yolo and other algorithms, to perform object detection on the cockpit images, and obtains the probability that the left-behind objects belong to the living body category. The server can also calculate the probability that the left-behind objects in the cockpit belong to the living body category through the vehicle terminal using an ecological sign sensor. Then the server compares the probability that the left-behind objects belong to the living body category with a threshold B. The threshold B represents a preset category threshold.

[0120] When the probability that the left-behind object belongs to the living category is greater than threshold B, it is determined that the left-behind object is a living being. The server sends a safety control instruction to the vehicle terminal, so that when the vehicle terminal receives the safety control instruction, it performs safety control on the corresponding vehicle of the vehicle terminal, such as opening the window, turning on ventilation, etc. Then the server sends the cockpit image to the remote control room for manual recheck. When the probability that the left-behind object belongs to the living category is less than threshold B, the server directly sends the cockpit image to the remote control room for manual recheck. When it is confirmed through manual recheck that the left-behind object in the cockpit image is a living being, the remote control room communicates with the passenger by phone through the customer service. When it is confirmed through manual recheck that there is no left-behind object in the cockpit image, the remote control room controls the corresponding vehicle of the vehicle terminal to continue operating. The remote control room returns the manual recheck result to the server, and the server generates a corresponding category label according to the manual recheck result, and uses the rechecked image as a training sample to update the parameters of the classification algorithm and the object detection algorithm.

[0121] When the server confirms that there is a left-behind object in the cockpit image and the passenger does not return to the vehicle within the allowable stay time stipulated by the traffic rules corresponding to the getting-off location, it sends a request for the extraction time at the getting-off location to the passenger. When the passenger returns the extraction time at the getting-off location through the mobile terminal within the preset specified time, it is judged whether the returned extraction time of the passenger exceeds the preset allowable extraction time. When the server detects that the returned extraction time of the passenger does not exceed the preset allowable extraction time and detects that the stay time of the vehicle at the getting-off location exceeds the allowable stay time stipulated by the traffic rules, it sends a waiting route to the vehicle terminal, so that the corresponding vehicle of the vehicle terminal circles around the getting-off location according to the waiting route and stops at the getting-off location at the extraction time. When the server detects that the passenger fails to return to the getting-off location to pick up the left-behind object after the timeout, the passenger fails to return the extraction time at the getting-off location through the mobile terminal within the specified time, or the extraction time returned by the passenger through the mobile terminal exceeds the allowable extraction time, it sends the service point location to the vehicle terminal to make the vehicle drive to the nearest service point, saves the left-behind object to the service point, and the server sends the service point location to the user mobile terminal. Then the server sends an operation instruction to the vehicle terminal.

[0122] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indication of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0123] Based on the same inventive concept, an embodiment of the present application also provides a legacy object processing device for implementing the above-mentioned legacy object processing method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following legacy object processing device can refer to the limitations on the legacy object processing method in the above text, and will not be repeated here.

[0124] In one embodiment, as Figure 7 shown, a legacy object processing device 700 is provided, including: a detection module 702, an identification module 704, and a prompt module 706, where:

[0125] The detection module 702 is configured to obtain a to-be-detected image, and input the to-be-detected image into a legacy object detection model for legacy object detection, so as to obtain the probability of the existence of a legacy object corresponding to the to-be-detected image;

[0126] The identification module 704 is configured to determine that there is a legacy object in the to-be-detected image based on the probability of the existence of the legacy object;

[0127] The prompt module 706 is configured to send a legacy object retrieval prompt to a target client, and the legacy object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position.

[0128] In one embodiment, the identification module 704 includes:

[0129] The legacy object determination unit is configured to determine that there is a legacy object in the to-be-detected image when it is detected that the probability of the existence of the legacy object meets a preset legacy object category recognition condition; or

[0130] when it is detected that the probability of the existence of the legacy object meets a preset recheck condition, send a legacy object recheck request to a control end, the legacy object recheck request carries the to-be-detected image, obtain the legacy object recheck result returned by the control end, and determine that there is a legacy object in the to-be-detected image based on the legacy object recheck result.

[0131] In one embodiment, the legacy object processing device 700 further includes:

[0132] A category recognition unit, configured to input the image to be detected into a legacy object classification model for classification and recognition, so as to obtain the degree of the target living body category of the legacy object corresponding to the image to be detected.

[0133] When the degree of the target living body category of the legacy object meets a preset vehicle control condition, send a safety control instruction to the vehicle side, so that the vehicle side performs safety control on the corresponding vehicle based on the safety control instruction.

[0134] Obtain the safety control result returned by the vehicle side, and send a legacy object living body re-inspection request to the control side, where the legacy object living body re-inspection request carries the image to be detected.

[0135] Obtain the target living body category confirmation result corresponding to the image to be detected returned by the control side.

[0136] In one embodiment, the legacy object processing device 700 further includes:

[0137] A training unit, configured to use the image to be detected as a training sample, and determine a legacy object existence label and a target living body category label based on the target living body category confirmation result;

[0138] Use the training sample and the legacy object existence label to update and train a legacy object detection model, so as to obtain an updated legacy object detection model;

[0139] Use the training sample and the target living body category label to update and train a legacy object classification model, so as to obtain an updated legacy object classification model.

[0140] In one embodiment, the prompt module 706 includes:

[0141] A speed calculation unit, configured to obtain a target extraction position and a target extraction time returned by a target client, where the target extraction position is selected from the candidate position set, and the target extraction time is selected from the retrieval times corresponding to each candidate position;

[0142] Obtain the current vehicle position corresponding to the vehicle side, and determine the current driving distance based on the current vehicle position and the target extraction position;

[0143] Calculate the driving speed corresponding to the vehicle side based on the current driving distance and the target extraction time;

[0144] Send a driving control instruction to the vehicle side, where the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle side travels at the driving speed and reaches the target extraction position at the target extraction time.

[0145] In one embodiment, the legacy object processing device 700 further includes:

[0146] A route generation unit, configured to obtain the arrival time of the vehicle end at the target extraction position, and calculate the waiting time corresponding to the vehicle end based on the arrival time and the target extraction time;

[0147] Obtain the available stay time corresponding to the target extraction position, and when it is detected that the waiting time is greater than the available stay time, generate waiting route information based on the waiting time and the target extraction position;

[0148] Send the waiting route plan to the vehicle end, so that the vehicle end controls the corresponding vehicle to drive according to the waiting route and drive to the target extraction position at the target extraction time.

[0149] In one embodiment, the legacy object processing device 700 further includes:

[0150] A driving unit, configured to obtain the service point position when no identity recognition request sent by the vehicle end is detected after a preset time period;

[0151] Send a driving instruction to the vehicle end based on the service point position, so that the vehicle end drives to the service point;

[0152] Generate legacy object position information based on the service point position, and send the legacy object position information to the target client.

[0153] Each module in the above-mentioned legacy object processing device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0154] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 8As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a legacy object detection model. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for processing legacy objects.

[0155] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 9 shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for processing legacy objects. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0156] Those skilled in the art can understand that Figures 8 - 9The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0157] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0158] Obtain an image to be detected, and input the image to be detected into a left-behind object detection model for left-behind object detection to obtain the existence probability of the left-behind object corresponding to the image to be detected; determine that there is a left-behind object in the image to be detected based on the existence probability of the left-behind object; send a left-behind object retrieval prompt to the target client, where the left-behind object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position.

[0159] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0160] Determining that there is a left-behind object in the image to be detected based on the existence probability of the left-behind object includes: when it is detected that the existence probability of the left-behind object meets the preset left-behind object category recognition condition, determining that there is a left-behind object in the image to be detected; or when it is detected that the existence probability of the left-behind object meets the preset re-inspection condition, sending a left-behind object re-inspection request to the control end, where the left-behind object re-inspection request carries the image to be detected, obtaining the left-behind object re-inspection result returned by the control end, and determining that there is a left-behind object in the image to be detected based on the left-behind object re-inspection result.

[0161] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0162] After determining that there is a left-behind object in the image to be detected when it is detected that the existence probability of the left-behind object meets the preset left-behind object category recognition condition, the following steps are further included: inputting the image to be detected into a left-behind object classification model for classification and recognition to obtain the degree of the left-behind object target living body category corresponding to the image to be detected; when the degree of the left-behind object target living body category meets the preset vehicle control condition, sending a safety control instruction to the vehicle end so that the vehicle end performs safety control on the vehicle corresponding to the vehicle end based on the safety control instruction; obtaining the safety control result returned by the vehicle end, and sending a left-behind object living body re-inspection request to the control end, where the left-behind object living body re-inspection request carries the image to be detected; obtaining the target living body category confirmation result corresponding to the image to be detected returned by the control end.

[0163] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0164] After obtaining the confirmation result of the target live object category corresponding to the image to be detected returned by the control end, it includes: using the image to be detected as a training sample, and determining the legacy object existence label and the target live object category label based on the confirmation result of the target live object category; using the training sample and the legacy object existence label to update and train the legacy object detection model to obtain an updated legacy object detection model; using the training sample and the target live object category label to update and train the legacy object classification model to obtain an updated legacy object classification model.

[0165] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0166] After sending a legacy object retrieval prompt to the target client, the legacy object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position, it further includes: obtaining the target extraction position and the target extraction time returned by the target client, the target extraction position is selected from the set of candidate positions, and the target extraction time is selected from the retrieval times corresponding to each candidate position; obtaining the current vehicle position corresponding to the vehicle end, and determining the current driving distance based on the current vehicle position and the target extraction position; calculating the driving speed corresponding to the vehicle end based on the current driving distance and the target extraction time; sending a driving control instruction to the vehicle end, the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle end travels at the driving speed and reaches the target extraction position at the target extraction time.

[0167] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0168] The method further includes: obtaining the arrival time of the vehicle end at the target extraction position, and calculating the waiting time corresponding to the vehicle end based on the arrival time and the target extraction time; obtaining the available stay time corresponding to the target extraction position, when it is detected that the waiting time is greater than the available stay time, generating waiting route information based on the waiting time and the target extraction position; sending the waiting route plan to the vehicle end, so that the vehicle end controls the corresponding vehicle to travel according to the waiting route and reaches the target extraction position at the target extraction time.

[0169] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0170] After sending a control instruction to the vehicle end based on the target extraction position and the target extraction time, so that the vehicle end controls the corresponding vehicle to reach the target extraction position at the target extraction time, it further includes: when the identity recognition request sent by the vehicle end is not detected after a preset time period, obtaining the service point position; sending a driving instruction to the vehicle end based on the service point position, so that the vehicle end travels to the service point; generating legacy object position information based on the service point position, and sending the legacy object position information to the target client.

[0171] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0172] Obtain an image to be detected, and input the image to be detected into a legacy object detection model for legacy object detection to obtain the probability of the existence of a legacy object corresponding to the image to be detected; determine that there is a legacy object in the image to be detected based on the probability of the existence of the legacy object; send a legacy object retrieval prompt to a target client, where the legacy object retrieval prompt includes a set of candidate locations and the retrieval time corresponding to each candidate location.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0174] Determining that there is a legacy object in the image to be detected based on the probability of the existence of the legacy object includes: when it is detected that the probability of the existence of the legacy object meets a preset legacy object category recognition condition, determining that there is a legacy object in the image to be detected; or when it is detected that the probability of the existence of the legacy object meets a preset recheck condition, sending a legacy object recheck request to a control end, where the legacy object recheck request carries the image to be detected, obtaining the legacy object recheck result returned by the control end, and determining that there is a legacy object in the image to be detected based on the legacy object recheck result.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0176] After determining that there is a legacy object in the image to be detected when it is detected that the probability of the existence of the legacy object meets a preset legacy object category recognition condition, the following steps are further included: inputting the image to be detected into a legacy object classification model for classification and recognition to obtain the degree of the living body category of the legacy object target corresponding to the image to be detected; when the degree of the living body category of the legacy object target meets a preset vehicle control condition, sending a safety control instruction to a vehicle end, so that the vehicle end performs safety control on the vehicle corresponding to the vehicle end based on the safety control instruction; obtaining the safety control result returned by the vehicle end, and sending a legacy object living body recheck request to the control end, where the legacy object living body recheck request carries the image to be detected; obtaining the target living body category confirmation result corresponding to the image to be detected returned by the control end.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0178] After obtaining the confirmation result of the target live object category corresponding to the image to be detected returned by the control end, it includes: using the image to be detected as a training sample, and determining the presence label of the remaining object and the target live object category label based on the confirmation result of the target live object category; using the training sample and the presence label of the remaining object to update and train the remaining object detection model to obtain an updated remaining object detection model; using the training sample and the target live object category label to update and train the remaining object classification model to obtain an updated remaining object classification model.

[0179] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0180] After sending a reminder for retrieving the remaining object to the target client, the reminder for retrieving the remaining object includes a set of candidate positions and the retrieval time corresponding to each candidate position, it further includes: obtaining the target extraction position and the target extraction time returned by the target client, the target extraction position is selected from the set of candidate positions, and the target extraction time is selected from the retrieval times corresponding to each candidate position; obtaining the current vehicle position corresponding to the vehicle end, and determining the current driving distance based on the current vehicle position and the target extraction position; calculating the driving speed corresponding to the vehicle end based on the current driving distance and the target extraction time; sending a driving control instruction to the vehicle end, the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle end travels at the driving speed and reaches the target extraction position at the target extraction time.

[0181] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0182] The method further includes: obtaining the arrival time of the vehicle end at the target extraction position, and calculating the waiting time corresponding to the vehicle end based on the arrival time and the target extraction time; obtaining the available stay time corresponding to the target extraction position, when it is detected that the waiting time is greater than the available stay time, generating waiting route information based on the waiting time and the target extraction position; sending the waiting route plan to the vehicle end, so that the vehicle end controls the corresponding vehicle to travel along the waiting route and reaches the target extraction position at the target extraction time.

[0183] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0184] After sending a control instruction to the vehicle end based on the target extraction position and the target extraction time, so that the vehicle end controls the corresponding vehicle to reach the target extraction position at the target extraction time, it further includes: when the identity recognition request sent by the vehicle end is not detected after a preset time period, obtaining the service point position; sending a driving instruction to the vehicle end based on the service point position, so that the vehicle end travels to the service point; generating remaining object position information based on the service point position, and sending the remaining object position information to the target client.

[0185] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the steps in the above method embodiments.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0187] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0188] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0189] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for handling legacy objects, characterized in that, the method includes: obtaining a to-be-detected image, and inputting the to-be-detected image into a legacy object detection model for legacy object detection to obtain the existence probability of the legacy object corresponding to the to-be-detected image; determining that there is a legacy object in the to-be-detected image based on the existence probability of the legacy object; sending a legacy object retrieval prompt to a target client, where the legacy object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position; obtaining a target extraction position and a target extraction time returned by the target client, where the target extraction position is selected from the set of candidate positions, and the target extraction time is selected from the retrieval times corresponding to each candidate position; obtaining the current vehicle position corresponding to the vehicle end, and determining the current driving distance based on the current vehicle position and the target extraction position; calculating the driving speed corresponding to the vehicle end based on the current driving distance and the target extraction time; sending a driving control instruction to the vehicle end, where the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle end travels at the driving speed and reaches the target extraction position at the target extraction time.

2. The method according to claim 1, characterized in that, the determining that there is a legacy object in the to-be-detected image based on the existence probability of the legacy object includes: when it is detected that the existence probability of the legacy object meets a preset legacy object category recognition condition, determining that there is a legacy object in the to-be-detected image; or when it is detected that the existence probability of the legacy object meets a preset re-inspection condition, sending a legacy object re-inspection request to a control end, where the legacy object re-inspection request carries the to-be-detected image, obtaining the legacy object re-inspection result returned by the control end, and determining that there is a legacy object in the to-be-detected image based on the legacy object re-inspection result.

3. The method according to claim 2, characterized in that, after the determining that there is a legacy object in the to-be-detected image when it is detected that the existence probability of the legacy object meets a preset legacy object category recognition condition, it further includes: inputting the to-be-detected image into a legacy object classification model for classification recognition to obtain the degree of the target living body category of the legacy object corresponding to the to-be-detected image; when the degree of the target living body category of the legacy object meets a preset vehicle control condition, sending a safety control instruction to the vehicle end, so that the vehicle end performs safety control on the vehicle corresponding to the vehicle end based on the safety control instruction; obtaining the safety control result returned by the vehicle end, and sending a legacy object living body re-inspection request to the control end, where the legacy object living body re-inspection request carries the to-be-detected image; obtaining the target living body category confirmation result corresponding to the to-be-detected image returned by the control end.

4. The method according to claim 1, characterized in that, the sending a legacy object retrieval prompt to a target client, where the legacy object retrieval prompt includes a set of candidate positions and the retrieval time corresponding to each candidate position, includes: Obtain the current vehicle position corresponding to the vehicle terminal, obtain each preset stoppable location on the map based on the current vehicle position, and use the each stoppable location as a candidate position set.

5. The method according to claim 1, wherein, the method further includes: Obtain the arrival time of the vehicle terminal at the target extraction position, and calculate the waiting time corresponding to the vehicle terminal based on the arrival time and the target extraction time; Obtain the stoppable time corresponding to the target extraction position, and when it is detected that the waiting time is greater than the stoppable time, generate waiting route information based on the waiting time and the target extraction position; Send the waiting route information to the vehicle terminal, so that the vehicle terminal controls the corresponding vehicle to travel according to the waiting route and travels to the target extraction position at the target extraction time.

6. The method according to claim 1, wherein, after sending the driving control instruction to the vehicle terminal, the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle terminal travels at the driving speed and travels to the target extraction position at the target extraction time, the method further includes: When the identity recognition request sent by the vehicle terminal is not detected after a preset time period, obtain the service point position; Send a driving instruction to the vehicle terminal based on the service point position, so that the vehicle terminal travels to the service point; Generate legacy object position information based on the service point position, and send the legacy object position information to the target client.

7. A legacy object processing device, wherein, the device includes: A detection module, configured to obtain a to-be-detected image, input the to-be-detected image into a legacy object detection model for legacy object detection, and obtain the existence probability of the legacy object corresponding to the to-be-detected image; An identification module, configured to determine that there is a legacy object in the to-be-detected image based on the existence probability of the legacy object; A prompt module, configured to send a legacy object retrieval prompt to the target client, where the legacy object retrieval prompt includes a candidate position set and the retrieval time corresponding to each candidate position; A speed calculation unit, configured to obtain the target extraction position and the target extraction time returned by the target client, the target extraction position is selected from the candidate position set, and the target extraction time is selected from the retrieval times corresponding to each candidate position; obtain the current vehicle position corresponding to the vehicle terminal, determine the current driving distance based on the current vehicle position and the target extraction position; calculate the driving speed corresponding to the vehicle terminal based on the current driving distance and the target extraction time; send a driving control instruction to the vehicle terminal, the driving control instruction carries the driving speed, so that the vehicle corresponding to the vehicle terminal travels at the driving speed and travels to the target extraction position at the target extraction time.

8. A computer device, including a memory and a processor, the memory stores a computer program, wherein, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images