VEHICLE AND METHOD FOR CHANGING A TRANSPORT CONFIGURATION BASED ON THE CONTENTS OF A VEHICLE'S STORAGE SPACE

Abstract

A method on a vehicle (210), comprising: driving the vehicle (210) to a customer location with an empty storage space (217) so that a customer at the customer location can return an item (321) to a store or warehouse; detecting an event purportedly changing the contents of a space on the vehicle (210); wherein the detection of an event purportedly changing the contents of a space on the vehicle includes detecting that an item (321) has been placed and left in the storage space (217) by a customer; monitoring the space by a camera (303) for disturbances relating to a background image applied to an interior surface of the vehicle space (501) after the event, wherein the background image includes a principal feature or pattern; determining, based on any monitored disturbances, whether the contents of the space are consistent with a defined event outcome;Driving the vehicle (210) to a shop or warehouse so that an item can be returned; and removing an item from the storage space (217) at the shop or warehouse.

Description

GENERAL STATE OF THE TECHNOLOGY Field of invention

[0001] This invention generally relates to the field of changing vehicle configurations and in particular to changing a vehicle configuration based on the contents of a vehicle's storage compartments. State of the art

[0002] Autonomous vehicles (AVs) can be equipped with various (and potentially secured) storage compartments that can be used for delivering and / or collecting items. For example, an autonomous pizza delivery vehicle might include a pizza warmer to keep pizzas warm during transport to a customer. Similarly, an autonomous grocery delivery vehicle might include a refrigerator, a freezer, and other storage space for other food items (perhaps for shopping bags) to prevent food from spoiling during transport to a customer. Likewise, an autonomous parcel delivery vehicle might include one or more storage compartments to hold packages during transport to a customer.

[0003] In other cases, an autonomous vehicle with a storage compartment is sent to a customer to accept a returned item. The customer can place the returned item in the storage compartment, and the autonomous vehicle can return to a designated location, such as a warehouse, a store, etc. List of characters

[0004] The specific features, aspects and advantages of the present invention can be better understood with reference to the following description and the accompanying drawings, in which the following applies: Fig. Figure 1 illustrates an example block diagram of a computing device. Fig. Figure 2 illustrates an exemplary computer architecture that enables network communication between an autonomous vehicle and other electronic devices. Fig. Figure 3A illustrates an exemplary computer architecture that facilitates the delivery of an item from a storage compartment of a vehicle. Fig. Figure 3B illustrates an exemplary computer architecture that facilitates the retrieval of an item into a storage compartment of a vehicle. Fig. Figure 4 illustrates a flowchart of an exemplary procedure for changing the configuration of an autonomous vehicle based on the contents of a vehicle's storage compartment. Fig. 5A illustrates an example vehicle that includes a vehicle compartment. Fig. 5B illustrates an example background pattern. Fig. 5C illustrates the exemplary background pattern of the Fig. 5B, which is an interior surface of the vehicle compartment Fig. 5A permeated. Fig. 5D illustrates an exemplary view of objects in the vehicle's interior. Fig. 5A on the exemplary background pattern of the Fig. 5B. Fig. Figure 5E illustrates an exemplary enlarged view of the objects of the Fig. 5D on the exemplary background pattern of the Fig. 5B. Fig. Figure 5F illustrates an exemplary image taken by a camera positioned above the exemplary pattern of the Fig. 5B in the vehicle compartment of the Fig. 5A is installed. Fig. 5G illustrates another exemplary view of an object in the vehicle's interior. Fig. 5A on the exemplary background pattern of the Fig. 5B. Fig. Figure 5H illustrates an exemplary enlarged view of the subject. Fig. 5G on the exemplary background pattern of the Fig. 5B. Fig. Figure 51 illustrates another exemplary image taken by the camera positioned above the exemplary pattern of the Fig. 5B in the vehicle compartment of the Fig. 5A is installed. DETAILED DESCRIPTION

[0005] The present invention relates to methods, systems, and computer program products for changing a vehicle's configuration based on the contents of its storage compartments. In some aspects, an autonomous vehicle is used to deliver an item. For example, an item can be placed in a vehicle's storage compartment, and the autonomous vehicle can then drive to a customer's location. At the customer's location, the customer can remove the item from the vehicle's storage compartment. After the item has been removed, the autonomous vehicle can return to a predetermined location, such as a store or warehouse.

[0006] In some scenarios, an autonomous vehicle is used to retrieve an item. For example, the autonomous vehicle can drive to a customer location with an empty cargo area. At the customer location, a customer can place the returned item in the vehicle's cargo area. After the item has been placed in the vehicle's cargo area, the autonomous vehicle can return to a designated location, such as a store or warehouse. At the designated customer location, an employee can then remove the item from the vehicle's cargo area.

[0007] Generally, it is advisable to ensure that vehicle storage compartments are indeed empty when expected to be empty, and actually contain a (suitable or correct) item when expected to contain the (suitable or correct) item. However, there may be no human in the autonomous vehicle. Therefore, the storage compartments of an autonomous vehicle can be electronically monitored both before and after the journey to a customer location, and before and after customer contact. For example, cameras mounted in the vehicle's storage compartments can be used to monitor the interior of these compartments.

[0008] For deliveries, a camera can be used to monitor the vehicle's storage compartment after a purported loading at a loading location (e.g., to confirm the presence of a delivery item in the vehicle's storage compartment) and after a purported unloading at a customer location (e.g., to confirm that the item has been picked up). For pickups, a camera can be used to monitor the vehicle's storage compartment before departure to a loading location (e.g., to confirm that the vehicle's storage compartment is empty) and after a purported loading at the customer location (e.g., to confirm the presence of a returned item in the vehicle's storage compartment).

[0009] In one aspect, a machine learning camera is used to monitor a vehicle's storage compartment. The machine learning camera is mounted inside the vehicle's storage compartment. An artificially generated background is applied to an interior surface of the vehicle's storage compartment (e.g., a surface on which items are placed for transport). The artificially generated background can include a key feature or a known pattern. The artificially generated background can be configured to help items stand out and reduce the likelihood of items blending into the artificially generated background.

[0010] The machine learning camera stores the artificially generated background, for example as a reference image, including the learning of specific features (e.g., one or more spectral, spatial, and temporal features) that can be used to characterize the background appearance of specific areas inside the vehicle's storage compartment. Image processing decision rules can be derived for background classification based on the main feature or known pattern.

[0011] The machine learning camera can detect changes or disturbances in the background caused by objects present on the surface (within the vehicle's storage compartment). A non-zero difference between the reference image and a current image of the artificially generated background can indicate a disturbance. Thus, an object in the foreground can be detected by classifying changes in the main background feature or a known pattern. After an object is detected in a vehicle storage compartment, a human can confirm whether the object is authorized. Detection of an unauthorized object can occur if a foreign object is present in a vehicle storage compartment, but the vehicle's storage compartment should be empty.After detecting a foreign or unauthorized object, the machine learning camera can provide images from inside the vehicle storage container to another computer system.

[0012] In one aspect, the computer system is located at an administrative center where a human can assess the location of the foreign object and / or its identity. If a foreign object is a shameful, dangerous, or risky item, the human can take precautions with an autonomous vehicle and notify the relevant authorities. If the foreign object belongs to a customer, the customer can be notified via text, email, or voice that the foreign object has been left in the vehicle's storage compartment. The human can also instruct the autonomous vehicle to remain at or return to a customer's location. A foreign object belonging to a customer could be a delivery item that the customer was unable to retrieve from the vehicle's storage compartment, or an item (e.g., mobile phone, keys, etc.) that the customer inadvertently placed in the vehicle's storage compartment.If the foreign object is a “disruptive” object, such as an empty bag or box, the human can enable the autonomous vehicle to return to a designated location (e.g., a store or warehouse, or another delivery location).

[0013] In one aspect, an artificially generated background lies within the visible light spectrum and is visible to the human eye. In another aspect, an artificially generated background lies outside the visible light spectrum and is not visible to the human eye. For example, the artificially generated background could be in the infrared spectrum (IR spectrum), ultraviolet spectrum (UV spectrum), etc.

[0014] Thus, a machine learning camera can monitor the vehicle's storage compartment at a loading point before delivery to confirm that the vehicle's storage compartment contains one or more items (e.g., pizzas, groceries, packages, boxes, bags, etc.) ready for delivery to the customer. After delivery, the machine learning camera can monitor the vehicle's storage compartment again to confirm that it is empty. If the vehicle's storage compartment is not empty, a human operator can be notified and take appropriate action.

[0015] Similarly, before picking up an item, the vehicle's storage compartment can be monitored to confirm that it is empty. If the vehicle's storage compartment is not empty, a human operator can be notified and take appropriate action, such as returning the autonomous vehicle to a warehouse or store for unloading. After a pickup from the customer, the vehicle's storage compartment can be monitored to confirm that it contains an (authorized) item for return. If the vehicle's storage compartment is empty, a human operator can be notified and take appropriate action. For example, the customer can be notified via text, email, or voice message that the returned item has not been placed in the vehicle's storage compartment.

[0016] In one aspect, when a customer returns an item, the vehicle's storage compartment is monitored after an item is placed in the vehicle's storage container. The machine learning camera can provide images from inside the vehicle's storage container to another computer system (e.g., an administrative control center). Based on these images, a human operator can confirm that the item placed in the storage container is the (authorized) returned item. If the item placed in the vehicle's storage compartment is not the (authorized) returned item and is otherwise non-critical, the customer can be notified via text, email, or voice message that the returned item was not the item placed in the vehicle's storage container.If the item placed in the vehicle's storage compartment is a shameful, dangerous, or risky item, the human operator can take precautions in the case of an autonomous vehicle and notify the relevant authorities.

[0017] Thus, electronic monitoring of vehicle storage containers generally facilitates changes to the configuration of the autonomous vehicle to ensure proper delivery and collection of items, address the use of vehicle storage spaces for nefarious purposes, and help recover items that have been unintentionally and / or improperly left in vehicle storage spaces.

[0018] Fig. Figure 1 illustrates an example block diagram of a computing device. 100 The calculating device 100It can be used to perform various operations, such as those discussed herein. The calculating device 100 It can function as a server, a client, or any other computing unit. The computing device 100 It can perform various communication and data transmission functions as described herein and can execute one or more application programs, such as those described herein. The computing device 100 can be any of a wide variety of computing devices, such as a mobile phone or other mobile device, a desktop computer, a notebook computer, a server computer, a portable computer, a tablet computer and the like.

[0019] The calculating device 100 includes one or more processor(s) 102 , one or more storage devices 104 , one or more interface(s) 106, one or more mass storage devices 108 , one or more input / output (I / O) devices 110 and a display device 130 , all of which went to a bus 112 are coupled. The processor(s) 102 includes one or more processors or controllers located in the storage device(s) 104 and / or the mass storage device(s) 108 Execute stored instructions. The processor(s) 102 They may also include various types of computer storage media, such as a cache memory.

[0020] The storage device(s) 104 includes / contain various computer storage media, such as volatile memory (e.g., random access memory - RAM) 114 ) and / or non-volatile memory (e.g., read-only memory - ROM) 116 The storage device(s)104 They may also contain rewritable ROM, such as flash memory.

[0021] The mass storage device(s) 108 Includes / contains various computer storage media, such as magnetic tapes, magnetic disks, optical disks, solid-state storage (e.g., flash memory), and so on. As in Fig. As shown in Figure 1, a specific mass storage device is a hard disk drive. 124 . Various drives can also be installed in the mass storage device(s) 108 This includes the ability to read from and / or write to various computer-readable media. The mass storage device(s) 108 includes removable media 126 and / or non-removable media.

[0022] The I / O device(s) 110includes / includes various devices that allow data and / or other information to be transferred to the computing device 100 can be entered or retrieved from it. (An) Example I / O device(s) 110 includes cursor control devices, keyboards, keypads, barcode readers, microphones, monitors or other display devices, loudspeakers, printers, network interface cards, modems, cameras, lenses, radars, CCDs or other image acquisition devices and the like.

[0023] The display device 130 includes any type of device capable of providing information to one or more users of the computing device 100 to display. Examples of the display device 130 include a monitor, a display terminal, a video projection device, and the like.

[0024] The interface(s) 106includes / contains various interfaces that the computing device 100 enable interaction with other systems, devices, or computing environments, as well as with people. (An) Example interface(s) 106 includes / include any number of different network interfaces 120 These include interfaces to Personal Area Networks (PANs), Local Area Networks (LANs), Wide Area Networks (WANs), wireless networks (e.g., Near Field Communication (NFC), Bluetooth, Wi-Fi networks, etc.), and the internet. Other interfaces include a user interface. 118 and a peripheral device interface 122 .

[0025] The bus 112 enables the processor(s) 102 , the storage device(s) 104 , the interface(s) 106, the mass storage device(s)108 and the I / O device(s) 110 , with each other as well as with other devices or components connected to the bus 112 are coupled, to communicate. The bus 112 represents one or more of different types of bus structures, such as a system bus, PCI bus, IEEE-1394 bus, USB bus, and so on.

[0026] Fig. Figure 2 illustrates an exemplary computer architecture 200 , which enables network communication between an autonomous vehicle 210 and other electronic devices. The autonomous vehicle 210 It can be a land-based vehicle with a variety of wheels, such as a car, van, light truck, etc., and can operate fully autonomously under almost all conditions. The autonomous vehicle 210It can be instructed to follow a route, drive to one or more destinations, etc., and can safely drive along roadways to move between locations as instructed. The autonomous vehicle 210 It may also include manual controls, allowing a driver to potentially control the autonomous vehicle. 210 can operate.

[0027] As shown, the autonomous vehicle includes 210 a vehicle-to-infrastructure (CI) interface 211 , a powertrain control system 212 , a brake control 213 , a steering system 214 , a calculating device 215 , sensors 216 and a storage space 217 The calculating device 215 can perform calculations for steering the autonomous vehicle 210 to operate autonomously. The computing device 215 can provide information regarding the operation, status, configuration, etc. of the autonomous vehicle210 and corresponding components from the sensors 216 received. The computing device 215 can make decisions regarding the control of the autonomous vehicle 210 based on information provided by the sensors 216 be received.

[0028] The sensors 216 A variety of devices can be used to monitor the operating components of the autonomous vehicle. 210 (e.g. tires, wheels, brakes, throttle, engine, etc.) to monitor an environment that the autonomous vehicle 210 surroundings (e.g., other vehicles, pedestrians, cyclists, static obstacles, etc.) and for monitoring the storage space 217 include the sensors 216 They can include cameras, LiDAR sensors, radar sensors, ultrasonic sensors, etc.

[0029] For example, a radar mounted on a front bumper (not shown) of the vehicle can 210is attached, a distance to the autonomous vehicle 210 to the next vehicle in front of the vehicle 210 Provide a sensor for a global positioning system (GPS) on the autonomous vehicle. 210 can provide geographical coordinates of the autonomous vehicle 210 provide. The data provided by the radar and / or other sensors 216 The provided distance(s) and / or the geographic coordinates provided by the GPS sensor can be used to control the autonomous operation of the autonomous vehicle. 210 to enable.

[0030] The calculating device 215 can be any of the devices relating to the computing device 100 The described components include the computing device. 215It can include programs for controlling vehicle components, including: brakes, drive (e.g., by controlling an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, air conditioning, interior and / or exterior lighting, etc. The computing device 215 can also determine whether it or a human operator controls the autonomous vehicle 210 controls.

[0031] The calculating device 215 It can communicate, for example, via a vehicle communication bus with other computing devices and / or controllers on the autonomous vehicle. 210 be coupled. For example, the computing device can 215 via a communication bus to the powertrain control 212 , the brake control 213 and the steering control 214 They must be coupled to monitor and / or control various corresponding vehicle components. In one aspect, the CI interface 211, the calculating device 215 , the sensors 216 , the powertrain control 212 , the brake control 213 and the steering control 214 as well as any other computing devices and / or controllers connected via a vehicle communication network, such as a Controller Area Network (CAN). The CI interface 211 , the calculating device 215 , the sensors 216 , the powertrain control 212 , the brake control 213 and the steering control 214 as well as any other computing devices and / or controllers can generate message-related data and exchange message-related data via the vehicle communication network.

[0032] The CI interface 211 A network interface can be used for wired and / or wireless communication with other devices over the network. 230 include the server computer 220and the mobile device 260 The user may also have network interfaces for wired and / or wireless communication with other devices over the network. 230 This includes any of the autonomous vehicles. 210 , the server computer 220 and the mobile device 260 the user and their respective components communicate with each other via the network 230 , such as a LAN, a WAN, or even the internet, be connected (or be part of one). Therefore, the autonomous vehicle can 210 , the server computer 220 and the mobile device 260the user's system, as well as any other connected computer systems and their components, generate message-related data and transmit message-related data (e.g., Near Field Communication (NFC) payloads, Bluetooth packets, Internet Protocol (IP) data packets, and other high-level protocols that use IP data packets, such as Transmission Control Protocol (TCP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), etc.) over the network. 230 exchange. In one aspect, the CI interface enables 211 in addition, vehicle-to-vehicle (CC) communication via ad-hoc networks between the autonomous vehicle 210 and other vehicles in the vicinity.

[0033] Fig. 3A illustrates an exemplary computer architecture 300 , which enables the delivery of an item from a vehicle's storage compartment. As in computer architecture 300As shown, the storage space includes 217 furthermore an area 301 and camera 303 (e.g., for machine learning). The area 301 is with a background 302 permeated. The camera 303 can the background 302 (e.g., as a reference image) and then save the storage space 217 on any disturbance of the background 302 Monitor. A disturbance, such as a non-zero difference between a current image of the area 301 and the reference image, the presence of an object in the storage space can be determined. 217 indicate.

[0034] As pictured, the item 321 (e.g. a package) currently in the storage space 217 included. Thus, the camera can 303 a background disturbance 302 recognize which indicates that an object is in the storage space 217 is included. The autonomous vehicle 210can be located at a person's location 313 be sent so that the person 313 the object 321 from the storage space 217 can remove. Following arrival at the site, the computing device can 215 recognize that the storage space 217 opened and then closed (e.g. via a contact sensor on a door, a lid, a top, etc. of the storage space). 217 ), allegedly to remove the object 321 to remove (a removal event). After the storage space 217 when closed, the camera 303 the storage space 217 any background disturbance 302 monitor.

[0035] In one aspect, the person 313 a customer and the item 321 delivered to the customer. In another aspect, the person 313 a worker and the object 321returned to a store, warehouse, or other return location.

[0036] It is possible that the camera monitoring 303 no background disturbances 302 recognizes. Therefore, the camera considers 303 the storage space 217 as empty. In response, the vehicle can 210 automatically drive to a designated location, for example, back to a warehouse or store (e.g., to pick up another package). Alternatively, the CI interface can be used. 211 a notification 331 to the central administration 311 send the central administration 311 notified that the storage space 217 It is empty. The human operator 312 can the notification 331 see. In response, the human operator can 312 instructions 332 to the vehicle 210 send the vehicle 210instruct to proceed to a designated location.

[0037] In another aspect, the camera recognizes 303 a background disturbance 302 , which indicates the presence of an object. In response, the CI interface can 211 a notification 331 to the central administration 311 send the central administration 311 notified that an item is in the storage space 217 was detected. The camera 303 can also images 334 the interior of the storage space 217 (via the C-1 interface) 211 ) to the central administration 311 send. The human operator 312 can the notification 331 received and the images 334 see.

[0038] The pictures 334 can demonstrate that the object 321 still in the storage space 217is included. In response, the human operator can 312 a message 333 (e.g. text, email, etc.) to the mobile device 314 send the message 333 can the person 313 notify that the item 321 in the storage space 217 was left behind. The human operator 312 can also provide instructions 332 to the vehicle 210 send. The instructions 332 can the vehicle 210 instruct them to return to or remain at that location, so that the person 313 the object 321 from the storage space 217 can remove.

[0039] The pictures 334 can alternatively represent that the object 322 in the storage space 217 is contained. The item 322 can be an object that the person 313 intentionally or unintentionally into the storage space 217laid down or left behind there.

[0040] The object 322 can be a personal part of the person 313 be, for example, a telephone (mobile device) 314 ) or key. In response, the human operator can 312 a message 333 to the mobile device 314 send. If the message 333 an email message, the message can 333 can also be received on other devices belonging to the person 313 are assigned. The message 333 can the person 313 notify that the item 322 in the storage space 217 was left behind. The human operator 312 can also provide instructions 332 to the vehicle 210 send. The instructions 332 can the vehicle 210 instruct them to return to or remain at that location, so that the person 313the object 322 from the storage space 217 can be seen.

[0041] The object 322 It could be a "disruptive" item, such as a leftover bag, box, or other packaging that is inconvenient for the item. 321 is assigned. In response, the human operator can 312 instructions 332 to the vehicle 210 send the vehicle 210 instruct to proceed to a designated location.

[0042] The object 322 It could be a dangerous or risky object (e.g., explosives, chemicals, etc.). In response, the human operator may 312 instructions 332 to the vehicle 210 send the vehicle 210 instruct the vehicle to proceed to a designated, safer location (e.g., away from other vehicles and people). The human operator 312may also notify authorities, including disclosing the identity and last known location of the person 313 .

[0043] It is also possible that several items will be delivered in the storage space. 217 are included. If fewer than all of the items are removed, the human operator can 312 the person 313 notify them to remove all remaining items.

[0044] Fig. 3B illustrates an exemplary computer architecture 350 , which facilitates the retrieval of an item into a vehicle's storage compartment. The vehicle 210 can be used with empty storage space 217 to a location of the person 343 be sent so that the person 343 an authorized item 361 into the storage space 217 can be laid. Following arrival at the site, the computing device can be installed. 215recognize that the storage space 217 opened and then closed (e.g. via a contact sensor on a door, a lid, a top, etc. of the storage space). 217 ), allegedly to remove the object 361 to insert (an insertion event). After the storage space 217 when closed, the camera 303 the storage space 217 a background disturbance 302 monitor.

[0045] In one aspect, the person 343 a customer and the item 361 returned by the customer. In another aspect, the person 343 an employee and the item will be 361 into the storage space 217 loaded for delivery to a customer.

[0046] It is possible that the camera monitoring 303 no background disturbances 302 recognizes. Therefore, the camera considers 303 the storage space 217as empty. In response, the CI interface can 211 a notification 371 to the central administration 311 send the central administration 311 notified that the storage space 217 is empty. The camera 303 can also images 374 the interior of the storage space 217 (via the CI interface) 211 ) to the central administration 311 send. The human operator 312 can the notification 371 received and the images 374 see.

[0047] In response, the human operator can 312 a message 373 (e.g. text, email, etc.) to the mobile device 344 send the message 373 can the person 343 notify that the storage space 217 It remains empty and that the authorized item 361 into the storage space 217to be inserted. The human operator 312 can also provide instructions 372 to the vehicle 210 send. The instructions 372 can the vehicle 210 instruct them to return to or remain at that location, so that the person 343 the authorized item 361 into the storage space 217 can insert.

[0048] In another aspect, the camera recognizes 303 a background disturbance 302 , which indicates the presence of an object. In response, the CI interface can 211 a notification 371 to the central administration 311 send the central administration 311 notified that the storage space 217 contains an object. The camera 303 can also images 374 the interior of the storage space 217 (via the CI interface) 211 ) to the central administration 311send. The human operator 312 can the notification 371 received and the images 374 see.

[0049] The pictures 374 can demonstrate that the authorized item 361 the only item that is in the storage space 217 is included. In response, the human operator can 312 instructions 372 to the vehicle 210 send the vehicle 210 instruct to proceed to a designated location, such as a delivery or return location.

[0050] The pictures 374 Alternatively, they can represent that the authorized item 362 in the storage space 217 is included (either alone or together with the authorized item) 361 ). The subject 362 can be an object that the person 343 intentionally or unintentionally into the storage space 217laid down or left behind there.

[0051] The object 362 can be a personal part of the person 343 be, for example, a telephone (mobile device) 344 ) or key, a wrong package, etc. In response, the human operator can 312 a message 373 to the mobile device 344 send. If the message 373 an email message, the message can 373 can also be received on other devices belonging to the person 343 are assigned. The message 373 can the person 343 notify that the item 362 from the storage space 217 as can be seen and that only the authorized item 361 into the storage space 217 to be inserted. The human operator 312 can also provide instructions 372 to the vehicle 210 send. The instructions 372can the vehicle 210 instruct them to return to or remain at that location, so that the person 343 the object 362 from the storage space 217 can extract and possibly the authorized item 361 into the storage space 217 can insert.

[0052] The object 362 It could be a "disruptive" item that is unauthorized but otherwise non-critical. If the item 362 is a “disruptive” object and the authorized object 361 not in the storage space 217 If contained, a reaction similar to the reaction when the object is present may occur. 362 a personal item. If, on the other hand, the object 362 is a “disruptive” object and the authorized object 361 also in the storage space 217 The human operator can 312 instructions 332 to the vehicle210 send the vehicle 210 instruct to proceed to a designated location, such as a delivery point or a return point.

[0053] The object 362 It could be a dangerous or risky item. In response to that (and depending on whether the item is authorized) 361 also in the storage space 217 (whether or not it is included) the human operator 312 instructions 372 to the vehicle 210 send the vehicle 210 instruct the vehicle to proceed to a designated, safer location (e.g., away from other vehicles and people). The human operator 312 may also notify authorities, including disclosing the identity and last known location of the person 343 .

[0054] Thus, an autonomous vehicle can generally detect an event that purports to change the contents of a vehicle compartment. For example, an autonomous vehicle can detect the opening and closing of a vehicle compartment (an event), purportedly to remove an item from the compartment or to place an item into the compartment. A machine learning camera can then monitor the vehicle compartment after the event for disturbances related to a (e.g., previously stored) background image applied to an interior surface of the vehicle compartment. For example, the machine learning camera can monitor a vehicle compartment for disturbances related to a background pattern after the vehicle compartment has been opened and closed.

[0055] Based on all monitored disturbances, the autonomous vehicle can determine whether the contents of the vehicle compartment correspond to a defined event outcome. For example, if the autonomous vehicle is making a delivery, it can determine, based on all monitored disturbances, whether the vehicle compartment is empty after it has been opened and closed. Similarly, if the autonomous vehicle is making a pickup, it can determine, based on all monitored disturbances, whether an object is present in the vehicle compartment after it has been opened and closed.

[0056] The autonomous vehicle can modify its configuration based on the given settings. Modifying the autonomous vehicle's configuration can include sending a notification to a central control unit, sending images to a central control unit, remaining at a location, returning to a previous location, driving to a new location, driving to a safer location, etc.

[0057] Fig. Figure 4 illustrates a flowchart of an example procedure. 400 to change the configuration of an autonomous vehicle based on the contents of a storage compartment in the vehicle. The procedure 400 This refers to the components and data of computer architectures 300 and 350 described.

[0058] The procedure 400This involves a camera storing a background image that is applied to an interior surface of a vehicle compartment on the vehicle ( 401 For example, the camera can 303 the background 302 save. The procedure 400 includes detecting an event that allegedly changes the number of items contained in the vehicle compartment ( 402 For example, the calculating device 215 recognize that the person 313 the storage space 217 opens and closes, ostensibly to remove the item 321 to remove (a removal event). Alternatively, the computing device can 215 recognize that the person 343 the storage space 217 opens and closes, ostensibly to remove the item 361 to insert (an insertion event).

[0059] The procedure 400This includes the camera monitoring the vehicle interior for any disturbance in relation to the background image after the event ( 403 For example, the camera can 303 the storage space 301 on any disturbance in relation to the background 302 monitor after the person 313 the object 321 allegedly removed. Alternatively, the camera can 303 the storage space 301 on any disturbance in relation to the background 302 monitor after the person 343 the object 361 allegedly filed a complaint.

[0060] The procedure 400 includes determining, based on the event and any monitored disturbance, whether the contents of the vehicle compartment are appropriate ( 404 For example, the calculating device 215 based on the fact that the person 313 allegedly the object 321removed (the removal event), and any monitored disturbance in the background 302 determine whether the contents of the storage space 217 is suitable or unsuitable. The calculating device 215 can access the contents of the storage space 217 consider it appropriate if the camera 303 the storage space 217 After the removal event, it is considered empty. On the other hand, the computing device 215 the contents of the storage space 217 consider it inappropriate if the camera 303 after the removal event, the presence of an object in the storage space 217 recognizes.

[0061] Alternatively, the computing device can 215 based on the fact that the person 343 allegedly the object 361 into the storage space 217 has been inserted (the insertion event), and any monitored disturbance in the background 302 determine whether the contents of the storage space217 is suitable. The calculating device 215 can access the contents of the storage space 217 consider it appropriate if the camera 303 the presence of an object in the storage space 217 after the insertion event (even though human confirmation based on the images is still possible). On the other hand, the computing device can 215 the contents of the storage space 217 consider it inappropriate if the camera 303 the storage space 217 deemed empty after the insertion event.

[0062] The procedure 400 This includes modifying the vehicle's configuration to respond to the determination (405). For example, the vehicle's configuration may 210 be modified to respond to a requirement that the storage space 217appropriately or inappropriately, it is empty, or appropriately or inappropriately, it contains an item. Modifying the vehicle's configuration 210 This can include sending a notification to a central administration, sending images to a central administration, remaining at a location, returning to a previous location, driving to a new location, driving to a safer location, etc. How the vehicle is configured 210 Whether it is modified may depend on the contents of the storage space. 217 whether it corresponds to an expected result or not.

[0063] For example, if the vehicle 210 Making a delivery involves recognizing that the storage space 217 If the system is empty after customer contact, this is an expected result. Therefore, changing the vehicle's configuration is not possible. 210 Directing the vehicle 210, to drive to a new location. On the other hand, recognizing that the storage space 217 If an item is still present after customer contact, this is an unexpected result. Therefore, changing the vehicle's configuration may be necessary. 210 Sending a notification and images to a central management office may be involved. Depending on whether the item belongs to the customer, is a "disruptive" item, or is a dangerous or risky item, the vehicle may be... 210 be instructed to remain at a location, return to a warehouse or store, or drive to a safer location.

[0064] The vehicle configuration 210 This can be similarly varied when an item is picked up, depending on the contents of the storage space. 217 whether it matches or does not match an expected result.

[0065] Fig. 5A illustrates an example vehicle 500 , which is a vehicle compartment 501 (e.g., a pizza oven) Fig. 5B illustrates an example background pattern 502 . Fig. 5C illustrates the exemplary background pattern 502 , which is an interior surface of the vehicle compartment 501 permeated. A machine learning camera (not shown) can be mounted on the top of the vehicle compartment. 501 be mounted and have a lens that points downwards onto the background pattern 502 is directed. The machine learning camera can detect the background pattern. 502 Save as a reference image indicating that the vehicle interior 501 is empty.

[0066] Fig. 5D illustrates an exemplary view of objects 511 and 512 (e.g. pizza boxes) in the vehicle compartment 501 on the background pattern 502 . Fig. Figure 5E illustrates an exemplary enlarged view of the objects 511 and 512 on the exemplary background pattern 502 . Fig. 5F illustrates a picture 522 , which was captured by the machine learning camera (not shown) positioned above the background pattern 502 on the top of the vehicle compartment 501 It is assembled. As shown, the items are 511 and 512 into the vehicle compartment 501 invited. The picture 522 The fault shows 521 , a non-zero difference between the reference image and an image of the objects 511 and 512 , which against the background 502 lie. The disturbance 521 indicates the presence of one or more items in the vehicle compartment 501 If a customer orders two pizzas, the vehicle would... 500the presence of one or more items in the vehicle compartment 501 This is expected. Therefore, no notifications or warnings will be sent to a central administration (e.g., back to a delivery service).

[0067] Fig. 5G illustrates an exemplary view of the subject. 511 in the vehicle compartment 501 on the background pattern 502 The object 511 It could be a pizza that was not originally taken by a customer. Fig. Figure 5H illustrates an exemplary enlarged view of the object. 511 on the exemplary background pattern 502 . Fig. 51 illustrates a picture 524 , which was captured by the machine learning camera (not shown). The image 524 The fault shows 523 , a non-zero difference between the reference image and an image of the object 511 , which is set against the background502 lies. The disturbance 523 indicates the presence of one or more items in the vehicle compartment 501 to.

[0068] Since the vehicle 500 If the customer is expected to take both of his pizzas, the disruption can occur. 523 Trigger a notification or warning to a central management system (e.g., return to the delivery service). The vehicle 500 It can also produce images similar to the Fig. 5H sends the data to central administration for evaluation by a human operator. Based on the images, the human operator can see that the customer placed a pizza in the vehicle compartment. 501 has left behind. The human operator can then notify the customer via text, email, or voice that a pizza is waiting for them in the vehicle compartment. 501 left behind. The human operator can also prevent the vehicle from 501The delivery location is left by the customer until the customer returns and picks up their other pizza.

[0069] In one aspect, one or more processors are configured to execute instructions (e.g., computer-readable instructions, computer-executable instructions, etc.) to perform any of a variety of described operations. The one or more processors can access information within the system and / or store information in system memory. The one or more processors can convert information between different formats, such as background features, background patterns, reference images, pictures, notifications, messages, instructions for autonomous vehicles, etc.

[0070] System memory can be coupled to one or more processors and can store instructions (e.g., machine-readable instructions, machine-executable instructions, etc.) that are executed by that one or more processors. System memory can also be configured to store any of a variety of other types of data generated by the components described, such as background features, background patterns, reference images, pictures, notifications, messages, instructions for autonomous vehicles, etc.

[0071] The preceding disclosure refers to the accompanying drawings, which form part of this document and illustrate specific implementations of the disclosure. It is understood that other implementations may be used and structural modifications made without deviating from the scope of this disclosure. References in the description to "an embodiment," "an exemplary embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or property, but not every embodiment necessarily includes that particular feature, structure, or property. Furthermore, such formulations do not necessarily refer to the same embodiment.Furthermore, it should be noted that if a particular feature, structure or property is described in connection with an embodiment, it is within the scope of the skilled person's knowledge to implement such a feature, structure or property in connection with other embodiments, whether this is expressly described or not.

[0072] Implementations of the systems, devices, and methods disclosed herein may include or utilize a specialized or general-purpose computer containing computer hardware, such as one or more processors and system memory, as discussed herein. Implementations within the scope of this disclosure may also include physical and other computer-readable media for transporting or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or specialized computer system. Computer-readable media on which computer-executable instructions are stored are computer storage media (devices). Computer-readable media that transport computer-executable instructions are transmission media.Thus, implementations of the disclosure may, for example, and without limitation, include at least two distinctly different types of computer-readable media: computer storage media (devices) and transmission media.

[0073] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), flash memory, phase-change memory (“PCM”), other types of storage, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.

[0074] An implementation of the devices, systems, and methods disclosed in this document can communicate via a computer network. A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted to or made available to a computer via a network or other communication link (either wired, wireless, or any combination thereof), the computer correctly views the link as a transmission medium. Transmission media can include a network and / or data links that can be used to transport desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.Combinations of the above should also be included within the scope of computer-readable media.

[0075] Computer-executable instructions include, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, a specialized computer, or a specialized processing device to perform a specific function or group of functions. Computer-executable instructions can be, for example, binary files, instructions in an intermediate format such as assembly language, or source code. Although the subject matter has been described in a specific language for structural features and / or methodological actions, it is understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary implementations of the claims.

[0076] The person skilled in the art will understand that the disclosure can be implemented in network computing environments with many types of computer system configurations, including dashboard or other vehicle computers, PCs, desktop computers, laptops, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure can also be practically implemented in distributed systems environments where both local and remote computer systems connected by a network (either by wired data links, wireless data links, or a combination of wired and wireless data links) perform tasks.In a distributed systems environment, program modules can reside in both local and remote storage devices.

[0077] Furthermore, the functions described in this document may be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and operations described in this document. Certain terms are used throughout the description and in the claims to refer to specific system components. Those skilled in the art will understand that components may be referred to by different designations. This document does not distinguish between components that differ in name but not in function.

[0078] It should be noted that the sensor embodiments discussed above may include computer hardware, software, firmware, or any combination thereof to perform at least some of their functions. For example, a sensor may include computer code configured to run on one or more processors and may include a hardware logic / electrical circuit controlled by the computer code. These exemplary devices are provided in this document for illustrative purposes and are not intended to be limiting. Embodiments of the present disclosure may be implemented in other types of devices, as is known to the person skilled in the art.

[0079] At least some embodiments of the disclosure relate to computer program products comprising such logic (e.g., in the form of software) stored on any computer-usable medium. Such software, when executed in one or more data processing devices, causes a device to operate as described in the present document.

[0080] Although various embodiments of the present disclosure have been described above, it is understood that these serve only as examples and not as limitations. It is evident to the person skilled in the art that various modifications in form and detail can be made without deviating from the spirit and scope of the disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above, but should be defined solely in accordance with the following claims and their equivalents. The preceding description has been provided for illustrative purposes. It makes no claim to completeness and is not intended to limit the disclosure to the specific form disclosed. Many modifications, variations, and combinations are possible in light of the foregoing teachings.Furthermore, it should be noted that any or all of the above alternative implementations can be used in any desired combination to form additional hybrid implementations of the disclosure.

Claims

[1] Procedures on a vehicle, comprising: Detecting an event that allegedly changes the contents of a space on the vehicle; Monitoring the space with a camera for disturbances related to a background image applied to an interior surface of the vehicle interior after the event; Determine, based on any monitored disturbances, whether the contents of the room correspond to a defined event outcome; and Modifying the vehicle's configuration based on the specification. [2] Method according to claim 1, further comprising that the camera stores the background image before recognizing the event. [3] Method according to claim 1, wherein the detection of an event that allegedly changes the contents of a space on the vehicle includes the detection of an event that allegedly removes an object from the interior of the space. [4] The method of claim 3, wherein determining whether the contents of the space correspond to a defined event outcome includes detecting the presence of an object in the space; and wherein modifying the configuration of the vehicle based on the determination includes: Sending a network communication to notify a computer system that the room contains the object; Receiving instructions from the computer system specifying how to proceed in order to enable the removal of the object from the room; and Operate the vehicle in accordance with the instructions received. [5] Method according to claim 1, wherein the detection of an event that allegedly changes the contents of a space on the vehicle comprises the detection of an event that allegedly places an object inside the space. [6] The method of claim 5, wherein determining whether the contents of the space correspond to a defined event outcome includes recognizing that the space is empty; and wherein modifying the configuration of the vehicle based on the determination includes: Sending a network communication to notify a computer system that the room is empty; Receiving instructions from the computer system specifying how to proceed in order to enable the collection of the item; and Operate the vehicle in accordance with the instructions received. [7] Procedure on a vehicle, the procedure comprising: Saving a background image by a camera that is applied to an interior surface of a vehicle compartment on the vehicle; Detecting an event that allegedly changes the number of items contained in the vehicle compartment; Monitoring the vehicle interior by camera for any disturbance in relation to the background image after the event; Determine, based on the event and any monitored disturbance, whether the contents of the vehicle compartment are appropriate; and Modifying the configuration of the autonomous vehicle to respond to the determination. [8] Method according to claim 7, wherein a camera storing a background image applied to an interior surface of a vehicle compartment comprises a machine learning camera that learns spectral, spatial and temporal features of the background image. [9] Method according to claim 7, wherein a camera storing a background image applied to an interior surface of a vehicle compartment comprises a machine learning camera storing a background image that includes a key feature or pattern. [10] Method according to claim 7, wherein a camera storing a background image applied to an interior surface of a vehicle interior comprises a machine learning camera storing a background image located outside the visible light spectrum. [11] Method according to claim 7, wherein the detection of an event that purportedly changes the number of items contained in the vehicle compartment comprises the detection of an event that purportedly removes all items from the interior of the vehicle compartment; where the monitoring of the vehicle interior by the camera for any disturbance in relation to the background image includes detecting a disturbance in relation to the background image by the camera after the event; where determining whether the contents of the vehicle compartment are suitable includes determining, based on the camera's detection of the disturbance, that the contents of the vehicle compartment are unsuitable; and Modifying the vehicle's configuration to respond to the determination includes the following: Sending a network communication to notify a computer system that the vehicle compartment contains the item; Receiving instructions from the computer system specifying how to enable the removal of the object from the vehicle compartment; and Operate the vehicle in accordance with the instructions received. [12] Method according to claim 11, wherein operating the vehicle according to the received instructions comprises driving the vehicle to a specified location. [13] Method according to claim 7, wherein the detection of an event that purportedly changes the number of items contained in the vehicle compartment comprises the detection of an event that purportedly places an item into the interior of the vehicle compartment; where the monitoring of the vehicle interior by the camera for any disturbance in relation to the background image includes the fact that the camera does not detect a disturbance in relation to the background image after the event; where determining whether the contents of the vehicle compartment are suitable includes determining, on the basis that the camera does not detect a disturbance, that the contents of the vehicle compartment are unsuitable; and Modifying the vehicle's configuration to respond to the determination includes the following: Sending a network communication to notify a computer system that the vehicle compartment is empty; Receiving instructions from the computer system specifying how to facilitate the collection of the item; and Operate the vehicle in accordance with the instructions received. [14] Method according to claim 13, wherein operating the vehicle according to the received instructions comprises driving the vehicle to a specified location. [15] vehicle, the vehicle comprising the following: a room with an interior surface that is interspersed with a background image; a camera mounted inside the room, a processor; and a system memory that is coupled to the processor and stores instructions configured to do the following: To cause the camera to save the background image; Causing the processor to detect an event that supposedly changes the number of objects contained in the room; To have the camera monitor the room after the event for any disturbance relating to the background image; To cause the processor to determine, based on the event and any monitored disturbance, whether the contents of the vehicle compartment are suitable; and Causing the processor to modify the vehicle's configuration to respond to the determination. [16] Vehicle according to claim 15, wherein instructions configured to cause the camera to save the background image include instructions configured to cause the camera to learn spectral, spatial and temporal features of a principal feature or pattern of the background image. [17] Method according to claim 15, wherein instructions configured to cause the processor to detect an event purportedly changing the number of items contained in the room include instructions configured to cause the processor to detect an event purportedly removing all items from the interior of the room; where instructions configured to cause the camera to monitor the room for any disturbances related to the background image include instructions configured to cause the camera to detect a disturbance related to the background image after the event; wherein the instructions configured to cause the processor to determine whether the contents of the space are suitable include instructions configured to cause the processor to determine that the contents of the vehicle compartment are unsuitable, based on the camera's detection of the disturbance and the event that purportedly removes all objects from inside the space; and wherein the instructions configured to cause the processor to modify the vehicle's configuration to respond to the determination include instructions configured to cause the processor to do the following: Sending a network communication to notify a computer system that the room contains the object; Receiving instructions from the computer system on how to remove the object from the room; and Operating the vehicle in accordance with the received instructions. [18] Vehicle according to claim 17, wherein instructions configured to cause the processor to operate the vehicle according to the received information include instructions configured to cause the processor to drive the vehicle to a specified location. [19] Method according to claim 15, wherein instructions configured to cause the processor to detect an event purportedly changing the number of items contained in the room include instructions configured to cause the processor to detect an event purportedly placing an item into the interior of the room; where instructions configured to cause the camera to monitor the room for any disturbances related to the background image include instructions configured to cause the camera not to detect a disturbance related to the background image after the event; wherein the instructions configured to cause the processor to determine whether the contents of the room are appropriate include instructions configured to cause the processor to determine that the contents of the room are inappropriate, based on the fact that the camera does not detect the disturbance and the event that purports to place an object inside the room; and wherein the instructions configured to cause the processor to modify the vehicle's configuration to respond to the determination include instructions configured to cause the processor to do the following: Sending a network communication to notify a computer system that the room is empty; Receiving instructions from the computer system on how to facilitate the collection of the item; and Operating the vehicle in accordance with the received instructions. [20] Vehicle according to claim 19, wherein instructions configured to cause the processor to operate the vehicle according to the received information include instructions configured to cause the processor to drive the vehicle to a specified location.

Images