Systems and methods for collaborative awareness

Abstract

A method, apparatus, and system are provided for collaborative classification of objects in an autonomous vehicle environment. A controller receives data quality indicators from a plurality of devices. Each data quality indicator reflects a quality of an instance of object sensing data held by the device. Each device determines its data quality indicator from its sensor data and an indication of an object boundary region. The controller makes a decision based at least in part on the data quality indicator, for example, as to which instances of object sensing data are used for an object classification task, which device is to perform the object classification task, any downsampling of sensor data to be performed prior to object classification, and decisions of resource allocation for object classification. The decision may be made by solving an optimization problem.

Description

Technical Field

[0001] The present invention generally relates to automatic object perception, and more particularly to a method and apparatus for classifying objects within a region of interest of an autonomous vehicle or the like. Background Art

[0002] Autonomous vehicles (AVs) in autonomous driving scenarios generally rely on basic real-time information of the surrounding environment, including various objects and environmental conditions. This allows the AV controller to maintain situational awareness, including identifying objects in the surrounding environment. Due to the limited field of view of a single vehicle, sufficient situational awareness may require obtaining information from other sources, such as other AVs or fixed sensors. Therefore, obtaining and processing such real-time information may require sharing a large amount of raw sensory data between AVs (vehicle-to-vehicle or V2V) and between AVs and fixed sensors or servers as part of vehicle edge computing (VEC). This may require a large amount of reliable communication capabilities to transmit data, and may require computing resources that may not be available at the AV to process such a large amount of sensory data while maintaining basic real-time situational awareness. To date, technical solutions proposed to promote situational awareness by sharing data between multiple AVs may be resource-inefficient or even impractical.

[0003] Therefore, it is desirable to provide a method and apparatus that obviates or mitigates one or more limitations of the prior art.

[0004] The purpose of the background art is to disclose information that the applicant believes may be relevant to the present invention. It is not necessary to admit, nor should it be construed, that any of the above information constitutes prior art against the present invention. Summary of the invention

[0005] An object of embodiments of the present invention is to provide a method, apparatus, and system for collaboratively classifying objects in an area of ​​interest of at least one target device (e.g., a target autonomous vehicle (AV)) through multiple devices.

[0006] In an embodiment, a controller may be used to receive object sensing data quality indicators determined by a plurality of devices including a target device. Each object sensing data quality indicator may indicate the quality of a corresponding instance of object sensing data, wherein the corresponding instance of the object sensing data is maintained by a corresponding device in the plurality of devices and indicates a corresponding object, the corresponding object belonging to a set of objects in a region of interest of at least the target device. The controller may be used to provide an indication of one or more instances of the instances of the object sensing data to be used in each of one or more object classification tasks to at least one device in the plurality of devices, each of the object classification tasks being used to classify a corresponding object in the set of objects. A system for collaboratively classifying objects is provided, the system may include the plurality of devices and the controller. In an embodiment, the device may then perform the object classification task.

[0007] In an embodiment, providing the indication of one or more of the instances of object sensing data may include: the controller providing a data selection indication to a specific device of the at least one device of the plurality of devices. The data selection indication may indicate that the instance of object sensing data held by the specific device and indicating a specific object is to be used in a specific corresponding object classification task in the object classification task, wherein the specific object belongs to the set of objects. The controller may select the specific device based at least in part on the following object sensing data quality indicator: an object sensing data quality indicator indicating the quality of the corresponding instance of the object sensing data held by the specific device.

[0008] In an embodiment, the controller may provide one or more task delivery instructions to at least one of the multiple devices. Each task delivery instruction may indicate that a specified object classification task in the object classification task will be performed at a specified device in the multiple devices. The task delivery instruction may be determined together with the following indication: the indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks. At least one of the task delivery instructions may include the following indication or be integrated with the following indication: the indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks.

[0009] In an embodiment, the controller may allocate resources for each of the one or more object classification tasks. The resources may include one or both of the following: computing resources, which will be used by the designated (one or more) devices in the multiple devices to support the associated one or more object classification tasks in the object classification task; communication resources, which will be used to communicate between the designated device in the multiple devices and another designated device in the multiple devices or between designated device pairs in the multiple devices to support the associated one or more object classification tasks in the object classification task. The controller may determine the resource allocation together with determining the following indication: the indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks. The resources may be allocated to at least partially meet: the combination of the following time will be at or below a predetermined threshold: the computing time to complete each of the associated one or more object classification tasks in the object classification task, and the communication time to communicate between the designated device in the multiple devices and other designated devices in the multiple devices to support each of the associated one or more object classification tasks in the object classification task. The controller may transmit an indication of the allocated resources to one or more of the plurality of devices, and the plurality of devices may use the allocated resources when performing the object classification task and avoid using more resources than the allocated resources when performing the object classification task. At least one of the plurality of devices may then perform the specified object classification task in the object classification task, for example, according to the task placement indication.

[0010] In an embodiment, the controller may provide one or more data parsing instructions to at least one of the plurality of devices. Each data parsing instruction may indicate that an associated instance of the instance of the object sensing data to be used in each of the one or more object classification tasks is to be downsampled to a specified degree before being used in the associated object classification task. The use of the associated instance of the instance of the object sensing data may include transmitting the object sensing data between members of the plurality of devices when necessary. The device may then downsample the object sensing data to the specified degree according to the data parsing instruction.

[0011] In an embodiment, the controller may allocate resources to minimize or limit a total resource consumption cost. The total resource consumption cost may include a weighted or unweighted sum of the computing resources to be used by each designated device in the plurality of devices to support all associated object classification tasks, and a weighted or unweighted sum of the communication resources to be used for communication between each designated pair of devices in the plurality of devices to support all object classification tasks.

[0012] In an embodiment, the at least one device of the plurality of devices may perform the object classification task according to an indication, an assignment, or both an indication and an assignment provided by the controller. Such indication may include an indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks.

[0013] In an embodiment, a boundary area module may be provided. The boundary area module may be used to determine a boundary area of ​​an object indicated by an instance of the object sensing data. Determining the instance of the object sensing data may include, for example, for each instance of the object sensing data, the boundary area module sending boundary area parameters of the boundary area to each of the plurality of devices. Each of the plurality of devices may use the boundary area parameters to determine a corresponding instance of the object sensing data from raw sensing data obtained by the device of the plurality of devices. The boundary area module is located at the controller or at least one of the plurality of devices.

[0014] In an embodiment, the indication of the one or more instances of the object sensing data may be provided to satisfy that an estimated classification accuracy of each of the object classification tasks is at or above a predetermined classification accuracy threshold. The estimated classification accuracy may be determined by the controller based at least in part on a corresponding object sensing data quality indicator for each object classification task in combination with a dimension of a boundary region of an object received by the controller from the boundary region module. The estimated classification accuracy of each object classification task may be indicated by the controller to one or more of the devices.

[0015] In an embodiment, the region of interest may be a region of interest of a plurality of target devices including the target device. The region of interest may be a union of sub-regions, wherein each sub-region is a region of interest of a corresponding target device in the plurality of target devices. The plurality of devices may include the plurality of target devices.

[0016] In an embodiment, at least the following indications may be determined based on context information: the indication of one or more of the instances of the object sensing data. The context information may include one or more of network topology context information, resource availability context information, and quality of service (QoS) requirement context information.

[0017] According to another embodiment of the present disclosure, a computer program product comprising instructions is provided. When the instructions are executed by a computer, the computer is caused to fully or partially implement any method disclosed herein.

[0018] Embodiments are described above in conjunction with various aspects of the present invention, and these embodiments can be implemented according to these aspects. It will be appreciated by those skilled in the art that embodiments can be implemented in conjunction with the aspects in which they are described, but can also be implemented together with other embodiments of this aspect. When the embodiments are mutually exclusive or incompatible with each other, this will be apparent to those skilled in the art. Some embodiments can be described in conjunction with one aspect, but can also be applied to other aspects, which will be apparent to those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Further, the features and advantages of the present invention will be readily understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A scenario of a system and method for classifying an object provided by an embodiment of the present disclosure is shown;

[0021] Figure 2 The embodiment of the present disclosure provides Figure 1 The boundary area of ​​the object;

[0022] Figure 3 The segmented boundary area provided by the embodiment of the present disclosure is shown;

[0023] Figure 4 The operation of supporting object classification provided by the embodiment of the present disclosure is shown;

[0024] Figure 5A The embodiment of the present disclosure provides Figure 1 object sensing data of a given object Object0 obtained by a given device AV0;

[0025] Figure 5B The embodiment of the present disclosure provides Figure 1 The object sensing data of Object0 obtained by the device AV0 and the device AV1;

[0026] Figure 6Components of a system for classifying an object provided by an embodiment of the present disclosure are shown;

[0027] Figure 7 The collaborative perception mechanism for classifying objects provided by the embodiment of the present disclosure is shown;

[0028] Figure 8 The device provided by the embodiment of the present disclosure is shown.

[0029] It should be noted that in all the drawings, the same features are identified by the same reference numerals. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure relate to methods, apparatus, and systems for classifying objects in a region of interest of a target device, such as a target autonomous vehicle (AV). The target device may alternatively be other mobile devices, such as, but not necessarily limited to, a robot or a handheld device (e.g., a smartphone). Therefore, some embodiments of the present disclosure relate to autonomous driving, but other embodiments may relate to other applications requiring object classification. The classification is performed collaboratively by multiple devices.

[0031] In an embodiment, the target device is an AV that needs to classify objects within its region of interest. The target device may be stationary. The target device may be moving. The target device may be accelerating or decelerating.

[0032] In an embodiment, a plurality of devices are provided These devices are associated with an area of ​​interest of at least one target device and include the (at least one) target device. Device n can be a mobile (e.g. AV) or stationary device connected to a (communication) network and having sensing capabilities, computing capabilities, or both to sense or detect objects (e.g. within a predetermined sensing range). The device can be a (e.g., stationary) roadside unit (RSU). The RSU can communicate with an edge server that provides computing resources. The RSU itself can include local computing resources. The device can be an edge server with communication resources. The device can be located within the area of ​​interest of the target device. The device can be located outside the area of ​​interest of the target device while being able to (e.g., having sufficient sensing range) sense objects within the area of ​​interest of (at least one) target device.

[0033] In an embodiment, devices (AV, other RSUs, edge servers) outside the communication range of the AV associated with a particular area of ​​interest may be included in the systems and methods of the present disclosure related to the following object classification: (for example) object classification within a particular area of ​​interest performed by one or more neighboring RSUs and / or ground stations capable of relaying information and / or data.

[0034] In an embodiment, a region of interest is a collection of objects A physical area, wherein the object set is subjected to the method disclosed in this article Classify. Object collection Includes at least one object m. Non-limiting examples of objects include autonomous vehicles (other than the target device), other motor vehicles (e.g., cars, scooters, trucks, trailers, agricultural equipment), emergency and road rescue vehicles, pedestrians, animals and wildlife, and non-motor vehicles (e.g., bicycles, skateboards). Objects can be in contact with the ground. Objects can be in the air (e.g., ice flakes, birds flying from another vehicle). Objects can be stationary or moving. Stationary objects can be roadside pylons, roadside safety tripods, and any other objects on the ground (e.g., roads, sidewalks, shoulders) in the area of ​​interest, etc. Classification can involve and be applicable to determining various characteristics of an object, such as its type, movement, size, shape, etc.

[0035] In an embodiment, the region of interest may be a region of interest of a plurality of target devices including the target device. Such a region of interest may be a union of sub-regions, wherein each sub-region is a corresponding region of interest of a corresponding target device in the plurality of target devices. The plurality of devices that obtain object sensing data and determine the object sensing data quality indicator may include such a plurality of target devices including the target device.

[0036] In an embodiment, the region of interest may be a three-dimensional (3D) geographic region, a geometric region (e.g., a cuboid, an ellipsoid), or a combination thereof. The region may have a uniform boundary, a non-uniform boundary, or a combination thereof. The region of interest may include a target device. The region of interest may be primarily or substantially located in front of the target device along the direction of movement of the target device. The region of interest may include regions on the sides and behind the target device, such as for detecting and classifying objects in the blind spot of the target device. The region of interest may be relative to the target device or multiple target devices, each target device requiring object classification within a corresponding sub-region of each target device. Adjacent sub-regions defining corresponding regions of interest may be adjacent, may be separated by gaps, may partially overlap, or may be a combination thereof. One or more smaller sub-regions may be included in a larger sub-region.

[0037] Figure 1An example scenario is shown in which at least one AV (i.e., target device) requires object classification. The scenario includes a target device AV0 110 in an area of ​​interest 105. A first AV AV1 121a, a second AV AV2 121b, and a third AV AV3 121c are also located within the area of ​​interest 105. A fourth AV AV4 121d is located outside the area of ​​interest 105, but close enough to it (or has a sufficient sensing range) to obtain sensing data of at least some objects in the area of ​​interest 105. Other AVs may be located at the border of the area of ​​interest (not shown).

[0038] like Figure 1 As further shown in FIG. 1 , a roadside unit (RSU) 122 is located outside the area of ​​interest. The RSU communicates with all AVs within its communication range, which for simplicity includes Figure 1 . RSU 122 also communicates with edge server 123. In one embodiment, the RSU providing relay function may not need to be connected to an edge server (not shown). In an embodiment, there may be zero, one, two or more RSUs. RSUs do not necessarily have to be connected to edge servers.

[0039] The target device AV0 110 needs to classify six objects in the region of interest 105. The region of interest 105 includes the AV and the road where the objects move along a direction of motion 106. The first object Object0 130 (e.g., a truck) is located in front of the target device AV0 110 in the direction of motion 106. The second object Object1 121 (e.g., a car) is located in front of the first object Object0 130 in the direction of motion 106. The third object Object2 132 (e.g., a cyclist) is located on the right shoulder of the road and can be stationary or moving. The fourth object Object3 133 (e.g., a pedestrian) is located on the left shoulder of the road and can be stationary or moving. The fifth object Object4 132 (e.g., a car) is located in front of the right of the target device AV0 110 in the direction of motion 106. The sixth object Object5 135 (e.g., a truck) is located in front of the left of the target device AV0 110 in the direction of motion 106.

[0040] In an embodiment, each device in the plurality of devices may obtain (eg, using one or more sensors thereof, such as a LIDAR sensor) raw sensing data D of an object or environment within its sensing range. n . Original data D nAll sensory data acquired by device n may be included. The raw sensory data is relatively unprocessed compared to other forms of sensory data discussed herein, however, the raw sensory data may have been processed to some extent. Based on its raw sensory data, device n may determine (e.g., obtain, extract, filter, select) object sensory data for each object m in the region of interest. (e.g., as long as the object is within the sensing range and is at least partially not blocked by other objects or devices). It should be noted that each specific AV in the region of interest or AV that can provide sensing data for the region of interest can directly share information about itself (e.g., related to the location, dimensions, travel speed, travel direction, planned travel path, etc. of the specific AV) with other devices (including other AVs). Therefore, in the context of the present disclosure, if direct sharing of self information between AVs is utilized, the object m in the region of interest can exclude these AVs. The object sensing data can use the corresponding boundary area parameters of each object m in the region of interest. The original sensing data of the device may include the original sensing data of all objects in the region of interest. For example, due to the limited sensing range of the device, the original sensing data of the device may not include the original sensing data of all objects in the region of interest. For example, the portion of all the original sensing data related to the points in the boundary region of the object m may be determined as the object sensing data. When LIDAR data is obtained by reflecting light off a surface at a certain point, the LIDAR data can be related to this point.

[0041] In an embodiment, using the object sensing data in the corresponding object classification task reduces communication resource consumption compared to using the raw data from each device. This is because only the object sensing data (or data representing the object sensing data for further processing or abstraction) is transmitted, rather than the entire raw sensing data. Selecting (or determining) one or more specific devices to provide object sensing data for the classification task of the corresponding object can be based at least in part on the quality of the corresponding object sensing data maintained by each specific device.

[0042] exist Figure 1In the exemplary embodiment shown, each AV may provide object sensing data for at least some of the six objects in the region of interest 105 within its respective sensing range, taking into account various obstacles (e.g., other devices, vehicles). For example, AV3 121c may provide object sensing data for: the front and right side of Object0 130; the front and right side of Object1 121; the front and left side of Object2 132; the front and right side of Object0 130; the right side of Object3 133 (the data may be of lower quality because Object3 133 is far from AV3 121c and is partially blocked by Object1 121); the front and left side of Object4 134; and the right side of Object5 135 (the data may be of lower quality because Object5 135 is partially blocked by Object1 121). In another example, AV4 121 d may not be able to provide any object sensing data for Object4 134 because Object4 134 is at least partially occluded by each of AV2 121 b , AV3 121 c , and Object1 121 .

[0043] Target device 0 110 may not be able to obtain any object sensing data of Object1 121 because it is occluded by Object0 130. However, object sensing data of Object1 121 may be provided by AV3 121c (front and right side of Object1 121), AV1 121a (back and left side of Object1 121), and AV2 121b (front and right side of Object1 121).

[0044] In an embodiment, for example, the spatial position of the object m may be represented by a 3D cuboid boundary region containing the object. The estimated boundary region parameters of such a boundary region are can be represented by a 9-tuple, Here, x m ,y m ,z m Specify the 3D position coordinates of the center of the cuboid area; Specify the length of the cuboid region along the x, y, and z axes (e.g. before rotation); Specifies the rotation angle of the cuboid region along the x, y, and z axes. m ,y m ,z m It can be defined in the global coordinate system and can be obtained by coordinate transformation between the local coordinate system and the global coordinate system.

[0045] Estimated boundary region parameters The bounding region may be received by each of the plurality of devices from a bounding region module (eg, a networked computing device or portion thereof) that is operable to determine or estimate a bounding region for each object in the region of interest, as discussed elsewhere herein.

[0046] Figure 2 An example of six objects in a region of interest is shown, each object having a corresponding 3D cuboid boundary region with a rotation angle of zero. Each of the cuboid boundary regions shown, namely Region0 230, Region1 221, Region2 232, Region3 233, Region4 234, and Region5 235, corresponds to an object, namely Figure 1 130 , Object1 121 , Object2 132 , Object3 133 , Object4 134 , and Object5 135 in the region of interest 105 of the target device 110 shown in .

[0047] In an embodiment, device n may determine (and maintain) for each object m (in the region of interest and detectable by device n) an object sensing data quality indicator The object sensing data quality indicator indicates that the corresponding object sensing data Object sensing data quality indicator It can be determined by the device, for example, by using the corresponding boundary area parameters of each object m The object sensing data determined by a device for an object in the region of interest and its corresponding quality indicator may inherently indicate that the object belongs to a set of objects in the region of interest of at least the target device. A device may use its own object sensing data to determine its own data quality indicator, thereby reducing the need to transmit the object sensing data to other devices for this purpose.

[0048] In an embodiment, the data quality indicator may capture the object sensing data in terms of number of points and spatial distribution, for example. The amount and diversity of data will affect the classification accuracy of the corresponding object classification task.

[0049] exist Figure 3 In the exemplary embodiment shown, the object m (e.g., Figure 1 The 3D cuboid boundary region (eg, Region0 230) of Object0 130 can be divided into K 3There are non-intersecting 3D cuboid boundary sub-regions (boundary sub-regions 230a to 230h), where K=2, K represents the partition resolution corresponding to the number of partitions along each axis. The lengths of each 3D cuboid boundary sub-region along the x, y, and z axes of the object m are respectively and Object sensing data quality indicators The kth (1≤k≤K 3 )element It can be expressed as (or proportional to or increasing function of) the following quantity: The number of observation points within the k-th cuboid boundary subregion among all observation points in . In other embodiments, the number of partitions along each axis of the cuboid is not necessarily equal. An observation point can be an instance of object sensing data associated with a given point in space, for example, resulting from LIDAR reflection from a surface at that given point.

[0050] In an embodiment, the boundary region module discussed herein may be located at or cooperatively coupled to a controller or one of the devices.

[0051] In an embodiment, the boundary region module may be used to receive or obtain abstract (or abstracted) sensing data from the devices. The abstract sensing data may be obtained by each device from the corresponding raw data D n The abstract sensing data may be obtained or determined from the raw sensing data, for example, by downsampling the raw sensing data, or by using an occupancy map of the raw sensing data, or by both. For example, each device may generate abstract sensing data from its raw sensing data, the abstract sensing data including the same general information as the raw sensing data but at a lower granularity or resolution. Thus, such abstract sensing data is less resource intensive to transmit, but can still be used to determine a boundary region representing an object in the region of interest.

[0052] In an embodiment, the boundary region module may fuse (e.g., combine, process) all instances of abstract sensory data received from all corresponding devices. Based on the abstract sensory data, the boundary region module generates and defines boundary region parameters (e.g., the 9-tuple discussed elsewhere herein). ). Each instance of the (estimated) boundary region parameters indicates the boundary of a regular (e.g., cuboid) spatial region that contains an object (or potential object) in the region of interest. Thus, the boundary region module can define the general position, size, and (possible) shape of the objects that will subsequently be classified. For example, boundary region detection can be performed by detecting clusters of spatial points in a light detection and ranging (LIDAR) point cloud or an abstract version thereof. For example, examples of boundary region detection are described in the prior art such as "Efficient L-shape fitting for vehicle detection using laser scanners" by Xiao Zhang et al., Proc. IEEE IV'17, pp. 54-59, 2017, and "A fast and accurate segmentation method for ordered LiDAR point cloud of largescale scenes" by Ying Zhou et al., IEEE Geosci. Remote. Sens. Lett., 11(11): 1981–1985, 2014.

[0053] In an embodiment, for example, in response to the device receiving a corresponding request from the controller, the object sensing data quality indicator can be transmitted (e.g., sent) to the controller. In another example, after determining (partial or full) the object sensing data quality indicator, the object sensing data quality indicator can be automatically (e.g., through pre-configuration of the device) transmitted to the controller. The controller can be a network controller. The controller can include features and functions understood by those skilled in the art. The controller can be used to enable (e.g., execute, promote) some or all features of the systems and methods disclosed herein.

[0054] In an embodiment, the controller may be deployed in or operably coupled to a network that enables communication between each of the devices and the controller. For example, the controller may be located in an RSU or an associated edge server. In some embodiments, the controller is located in an AV, such as a target device.

[0055] In an embodiment, a system may include a controller and a group of devices, including a target device. The devices in the group of devices and the controller communicate with each other, for example, via a wireless communication link between them. The devices and the controller in the system may be configured according to various features and methods of the present disclosure.

[0056] In an embodiment, a controller that receives object sensing data quality indicators from a plurality of devices may provide the object sensing data to each such device (or at least one such device). Such an indication may indicate: object sensing data One or more of the instances of the one or more object classification tasks are to be used in each of the one or more object classification tasks for classifying a corresponding object in the object set. Such an indication may be sent by the controller to each (or at least one) device and may indicate to the device the object sensing data it holds. Which of the instances are to be used for the classification task of the object. Thus, the controller may use the received data quality indicators to determine the object sensing data instances to be used for a given object classification task.

[0057] exist Figure 4 In the exemplary embodiment shown in FIG. 1 , each device n100 in the plurality of devices obtains and maintains sensing data of an object m. Each device n100 also determines an object sensing data quality indicator indicating the quality of the corresponding instance of the object m sensing data. 180, and the indicator 180 is transmitted to the controller 300. The controller 300 determines the object sensing data to be used in each of the one or more object classification tasks. The controller may send a corresponding one or more instances of the ...

[0058] In an embodiment, the controller provides (object sensing data) to the device n. ) instance is based (at least in part) on a data selection decision The data selection decision includes selecting or determining, by the controller, which instances of object sensing data held by the respective devices are to be used in the respective object classification tasks. The controller may then send corresponding data selection indications to the devices, indicating to each device n the instances of object sensing data for its object m. will be used in the corresponding object classification task for object m. The data selection decision (or its indication) can be expressed as The binary decision variables Indicates object sensing data held by device n (instance of) for task m, otherwise

[0059] For example, Figure 5A and Figure 5B In the illustrative embodiment shown in FIG. 1 , a first device as a target device (AV0 110) can provide object sensing data (170a) (e.g., ), the object (Object0130) only has a boundary region (Region0230) behind the object. This is because the target device AV0110 is immediately behind the object Object0130. Such limited object sensing data (170a) may not be sufficient to classify the object accurately enough (e.g., above a predetermined classification accuracy threshold). However, Figure 1 Other devices such as AV1 121a shown in FIG. 121a may provide object sensing data (i.e., 170b, 170c, collectively referred to as 170b, 170c, and ... ). Using the instance of object sensing data provided by different devices for the same object helps to improve the classification accuracy of the corresponding object classification task. Figure 5B As shown, AV1 121a may provide object sensing data 170c in front of Object0 130 and other object sensing data 170b to the left of Object0 130. In various embodiments, the controller may provide the classification accuracy estimate to one, some, or all of the devices.

[0060] In an embodiment, the classification accuracy of the classification task of object m may be determined (or estimated) using, for example, a pre-trained deep neural network (DNN) model or other machine learning techniques. Using such machine learning, the classification accuracy of a classification task using selected (e.g., candidate) instances of object sensing data (as represented in the corresponding data selection decision) can be determined. Taking the DNN modeling approach as an example, the classification accuracy can be expressed herein as in, is the dimension of the bounding region of object m received by the controller from the bounding region module, The fused object sensing data quality indicator representing all corresponding selected instances of object sensing data depends on the data selection decision s and the optional data parsing decision g. A requirement may be imposed on the classification accuracy such that Among them, A is the predetermined classification accuracy threshold. It should be noted that the classification accuracy defined here is the same as the 9-tuple Six of the nine values ​​in are irrelevant, thus simplifying the DNN model.

[0061] Therefore, the classification accuracy of an object can be estimated using techniques such as machine learning based on the dimensions of the boundary region of the object and further based on data quality indicators. For example, when the data quality indicator indicates that a sufficient number of data points are available and are sufficiently well distributed throughout the boundary region, the classification accuracy can be considered relatively high. A larger number of data points and a good (e.g., uniform) distribution can lead to higher classification accuracy. Using these values ​​and some appropriate training examples, a machine learning model can be generated that can accurately predict object classification accuracy. Other heuristic techniques or functions may be used instead of machine learning to predict classification accuracy.

[0062] In an embodiment, the controller may select a particular device n to provide its object sensing data for a particular object classification task. (i.e., data for object m held by the device n). The selection may be based at least in part on the object sensing data. The quality of the instance is indicated by the corresponding object sensing data quality indicator

[0063] For example, as previously discussed Figure 1 As discussed, the quality of the object sensing data of Object1 121 maintained by AV4 121d may be low (or AV4 121d may not be able to obtain any sensing data of Object1 121) because Object1 121 is substantially blocked outside AV4 121d by AV2 121b. (This assumes that the sensor of AV4 121d cannot penetrate AV2 121b to obtain the sensing data of Object1 121.) The controller may determine that the classification task of Object1 121 does not require low-quality object sensing data (as represented by the corresponding object sensing data quality indicator transmitted to the controller by AV4 121d). Therefore, instances of object sensing data with low object sensing data quality indicators can be omitted from the classification task of the corresponding object, and the low object sensing data quality indicator captures the corresponding sensing data volume, data diversity, or both (for example, it can be represented according to the number of points and spatial distribution of sensing data points). This reduces communication and computational overhead.

[0064] In an embodiment, a controller may determine or select a resolution of an instance of object sensing data. For example, a device that is in close proximity to an object to be classified may obtain higher quality object sensing data than another device that is farther away from the object. Transmitting higher quality data between devices typically requires more communication resources (e.g., bandwidth, time to transmit data), which may result in the object classification task taking longer. Such higher quality data is not necessarily required to classify the object with sufficient accuracy (e.g., predetermined by an accuracy threshold). In this case, the controller may select or determine the resolution of some or all instances of object sensing data for some or all object classification tasks, which may be represented by the controller as a data resolution decision. The data resolution decision may be represented as in, represents the ratio between the number of observation points in a specific instance of the object sensing data of object m held by device n after and before data downsampling on the device n side for object m. Therefore, for at least one object and at least one device, the controller may determine that the object sensing data of the object and held by the device is downsampled by a specified amount before use (including data transmission (if applicable) and data processing). It should be noted that the controller may determine not to downsample any object sensing data. For example, downsampling may involve merging (e.g., averaging) a plurality of data points together or filtering out some of the data points (in the case where these data points indicate locations that are physically very close to each other). For example, in a rectangular grid of data points, each data point indicates a LIDAR return signal for a 1 cm×1 cm area, the data points may be downsampled by merging a subgrid of 10×10 data points to produce a single data point indicating an average LIDAR return signal for a 10 cm×10 cm area, or by filtering out a portion of the 10×10 subgrids of data points (which may be randomly selected or selected according to a predetermined rule or method). Downsampling may involve lossy data compression of the object sensing data to produce a version of the object sensing data that contains less information but is generally representative of the characteristics of the original object sensing data.

[0065] In one embodiment, data resolution may be selected for all instances of object sensing data maintained by all devices.

[0066] In another embodiment, data resolution may be selected for some instances of object sensing data and for some or all devices. For example, this may be based at least in part on the corresponding object sensing data quality (or an indicator thereof) of the instances of object sensing data. For example, one or more corresponding data quality metrics may be provided. If the data quality (or an indicator thereof) of a particular instance of object sensing data does not meet a predetermined data quality metric, the controller may select (as part of a data resolution decision) a lower data resolution for the particular instance (e.g., by indicating the degree of downsampling of the object sensing data) to reduce the consumption of communication resources when transmitting the particular instance of object sensing data between devices (while also allowing the classification task of the object requiring the particular instance of object sensing data to have sufficient accuracy). In another example, the controller may (at least in part) select a lower data resolution for a particular instance based on an estimated communication time to transmit the particular instance (or all instances to be transmitted) to a device selected for performing a corresponding object classification task that exceeds a predetermined data transmission threshold (or predetermined communication time). Therefore, data resolution may be reduced as long as other object classification goals are still met. This has the effect of allowing appropriate object classification to occur in a resource-efficient and timely manner.

[0067] In an embodiment, the controller may provide a data parsing indication to one or more devices maintaining a corresponding instance of object sensing data based on the data parsing decision. Such an indication may indicate to the device to downsample its corresponding instance of object sensing data by a specific degree (e.g., ratio) indicated by the received indication before using the corresponding instance (e.g., and before transmitting the corresponding instance to a device selected for performing an associated object classification task).

[0068] In an embodiment, the controller may select or determine a specific device to perform the classification task of a specified object. Indicate multiple devices Device n′ in the set performs the classification task of the object m. Equal to set The set of sensor devices may be different from the set of computing devices. Such a task placement decision (or its indication) may be expressed, for example, as Among them, the binary decision variables Indicates that the classification task of object m will be placed on computing node n′, otherwise The controller may make such determinations for some or all designated objects in the region of interest. Thus, the controller designates certain devices to classify certain objects.

[0069] In an embodiment, the controller may also provide one or more task placement instructions to at least one of the multiple devices based on the task placement decision. Each task placement instruction may indicate that a specified object classification task in the object classification task will be performed at a specified device in the multiple devices. Such a task placement decision (and its corresponding indication) may be determined together with other decisions, such as a data selection decision (and its corresponding indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks).

[0070] In some embodiments, the primary decision may be task placement, while other decisions (e.g., data selection) are not necessarily made by the controller. For example, at least one of the task placement instructions may include or be integrated with an indication of one or more of the instances of object sensing data to be used in each of the one or more object classification tasks.

[0071] For example and reference Figure 1 As part of the data selection decision for the classification task of Object0 130, the controller may identify (based at least in part on the corresponding object sensing data quality indicators) four devices: device AV0 110, device RSU 122, device AV1 121a, and device AV3 121c. To some extent, as part of the data selection decision, the controller may specify and are all set equal to 1. Each of the four selected devices will provide four instances of corresponding object sensing data to Object0 130 for use in the classification task of Object0 130. (The devices that perform this classification task by receiving and processing these instances of object sensing data may be selected in a task placement decision, which may also be made by the controller, as described below.)

[0072] In one embodiment, the controller may send an indication of each of the data selection decisions to respective ones of the four devices, indicating that their respective instances of the object sensing data of Object0 130 are to be used in the classification task of Object0 130. Additionally or alternatively, the controller may send such indications to the devices selected to perform the classification task of Object0 130.

[0073] In an embodiment, the controller may indicate to the device that the device is to perform (one or more) classification tasks for a specified object (or object classification task). For example, the controller may make a task placement decision such as As shown, device AV0 110 is selected for the classification task of Object0 130 .

[0074] In one embodiment, the controller may send a corresponding indication of the task placement decision to each of the four devices, the indication indicating to these devices where to send the corresponding four instances of the object sensing data of Object0 130, that is, to send the corresponding four instances of the object sensing data of Object0 130 to the device AV0 110. Such an indication received by each of the four devices may also include (or be equivalent to) an indication of a corresponding data selection decision, the indication of the data selection decision indicating to each of the four devices that the corresponding retained instance of the object sensing data will be used in the classification task of Object0 130. Such an indication of the task placement decision received by each of the four devices (or at least one device) may also indicate to at least one of the four devices that it will provide the instance of the object sensing data of Object0 130 it retains to the device selected for the classification task of Object0130. For example, referring to the above Figure 1 In the discussion, the device RSU 122, the device AV1 121a, and the device AV3 121c receive a task placement indication indicating that the classification task of Object0 130 is to be performed by the device AV0 110. In response, these devices may send corresponding instances of the object sensing data of Object0 130 to the device AV0 110. In other words, the task placement indication received by the device itself may include or imply a data selection indication indicating to the device that the object sensing data it holds will be used in the classification task of the corresponding object. In this case, if the device does not receive a specific task placement indication, then by default, the corresponding object sensing data held by the device may not be used in the classification task of the object.

[0075] Additionally or alternatively, the controller may send an indication of a task placement selection decision to the selected device AV0 110, indicating that it will perform the classification task of Object0 130. In another embodiment, if device AV0 110 receives such an indication in addition to receiving the above-mentioned indication of the data selection decision regarding the four devices (AV0 110, RSU 122, AV1 121a, AV3 121c), then device AV0 110 may send a corresponding request to each of the four devices except itself, requesting that a corresponding instance of the object sensing data be sent to AV0 for use in the classification task of Object0 130. AV0 also uses its own local data for the classification task. It is typical but not an absolute requirement that a device performing a classification task will use its own local data in such a task. Devices using their own local data in classification tasks can generally reduce data transmission and associated delays.

[0076] In another example, and referring to the above Figure 1 As discussed above, a controller may send a task placement indication to all devices selected to perform at least one object classification task. In this case, each device (or at least one device) receives information indicating the device that will perform the specified classification task. Such a task placement indication does not necessarily include a data selection indication, which may be sent by the controller separately to the device (as well as indicating which instances of object sensing data are selected for use in the classification task for the corresponding object). In other words, the task placement indication and the data selection indication may be sent as separate indications, although the corresponding decisions (task placement and data selection) may be determined together by the controller.

[0077] In one embodiment, as described above, if an instance of object sensing data to be used for a classification task for a specific object is held by a device also selected to perform the classification task for the specific object, transmission of the object sensing data may be omitted because such data is already held by the same device.

[0078] In an embodiment, the classification task for each object in the region of interest may be assigned by the controller to one or more devices, depending at least in part on the number of devices available to perform at least one object classification task. The controller may facilitate direct sharing (or sending, transmitting, communicating) of object sensing data between devices that maintain corresponding instances of object sensing data and devices selected by the controller to perform classification tasks (the classification tasks require corresponding instances of the object sensing data), rather than collecting the object sensing data and sending it to the selected node to perform the object sensing data. Thus, devices may transmit data to each other directly or through an intermediate device, which may be a controller or other one or more devices. Devices may perform one, two, or more object classification tasks.

[0079] In an embodiment, a controller may allocate (e.g., determine, allocate) resources for each of one or more object classification tasks associated in an object classification task. The allocated resources may be resources used by a device to perform the object classification task, such as computing resources, communication resources, or a combination thereof. The controller may transmit an indication of the allocated resources to a device performing the object classification task, a device providing object sensing data to be used in the object classification task, other communication infrastructure devices, or a combination thereof. The device may then use the allocated resources when performing the object classification task while avoiding using more than the allocated resources when performing the object classification task.

[0080] In an embodiment, the allocated resources may include computing resources α to be used by the selected device n′. n′ , to support (eg, perform) one or more object classification tasks associated with the object classification task to be performed at device n′. Computing resource α n′ It can be represented as a portion of the total computing resources of device n' that can be used for object classification. The computing resources can include processor time allocation, number of processors, number of threads, memory or cache allocation, etc., or a combination thereof.

[0081] In one embodiment, the computing resource α n′ may be allocated for all object classification tasks to be performed at the selected device n'. In one embodiment, when the selected device performs multiple object classification tasks, a portion of the device's computing resources may be allocated for each such task.

[0082] In an embodiment, the allocated resources may include communication resources β to be used for communication (e.g., transmission of the corresponding one or more instances of the object sensing data) between the device n holding the corresponding (one or more) instances of the object sensing data and the device n′ selected to perform the (one or more) classification tasks of the corresponding (one or more) objects. n,n′ The communication resource β may be determined for a given individual transmission of a single instance of object sensing data. n,n′ The communication resource β n,n′ A total transmission of all (or at least one) instances of object sensing data between two specified devices or between a selected pair of devices, such as a device providing an instance of object sensing data and a device performing object classification based at least in part on the instance of object sensing data, may be determined. Communication resources may be determined for some or all communications required between two or more devices performing one or more object classification tasks. Communication resources may include spectrum resources, such as channels, spreading code groups, time slots or frequency bands, etc., or a combination thereof.

[0083] For example, refer to Figure 1 , assuming that device AV0 110 is selected to perform the classification task of Object0 130 and Object4 134. It is further assumed that the respective instances of the object sensing data for the classification task of Object0 130 will be provided by device AV0 110, device RSU 122, device AV1 121a, and device AV3 121c. It is further assumed that the respective instances of the object sensing data for the classification task of Object4 134 will be provided by device AV0 110, device AV1 121a, and device AV3 121c. In this case, the communication resource β AV1，AV0 The object sensing data may be distributed for the corresponding object classification task from the device AV1 121a to the device AV0 110 and The total transmission of the instance. Similar communication resources can be allocated for transmitting the object sensing data from AV3 to AV0. AV0 does not need to communicate its own object sensing data to itself (through an external channel). The resource allocation decision can be expressed as and Among them, α n′ represents a portion of the computing resources at the device n′ allocated for use when performing one or more object classification tasks at or by the device n′; β n,n′ Represents a portion of the bandwidth allocated to a communication link (or for transmitting a corresponding one or more instances of object sensing data) from a device n holding a corresponding (one or more) instance of object sensing data to a device n' selected to perform a (corresponding one or more) classification task for a corresponding (one or more) object. As described above, resource allocation decisions are made for devices and are independent of tasks. In other embodiments, task-specific resource allocation decisions may be made.

[0084] In an embodiment, the controller may determine the resource allocation along with data selection decisions and corresponding indications thereof for one or more of the instances of object sensing data to be used in each of the one or more object classification tasks.

[0085] It is also important to note that communication and computational resource allocation decisions can be made in conjunction with data parsing decisions. Downsampling data requires fewer resources for communication and computation, so resource allocation can be adjusted accordingly.

[0086] In one embodiment, the controller may allocate resources to (at least partially) minimize or limit (e.g., at or below a predetermined threshold) the (e.g., expected or estimated) computation time for each of the one or more object classification tasks associated in the object classification task to be completed by (or by) a designated (or selected by) device. In another embodiment, the controller may allocate resources to (at least partially) minimize or limit (e.g., at or below a predetermined threshold) the (e.g., expected or estimated) communication time for communication between the designated device and one or more other devices selected to provide corresponding instances of object sensing data to the designated device (to support each of the one or more object classification tasks associated in the object classification task). In another embodiment, the controller may allocate resources (at least partially) to minimize or limit both computation time and communication time, for example as a combination of a weighted sum or an unweighted sum, etc.

[0087] In an embodiment, multiple devices may simultaneously communicate with an AV selected for one or more of the object classification tasks (e.g., provide corresponding instances of their object sensing data) by using other transmission schemes such as Non-Orthogonal Multiple Access (NOMA).

[0088] For example, the computation time may depend on the computational requirements of each object classification task. The (e.g., estimated) computational requirements or requirements of the classification task for object m may be expressed as μ (m) denoted by , and expressed in CPU cycles or other appropriate metrics (e.g., measured, estimated, evaluated). The computational intensity (expressed in CPU cycles / point) denoted as ∈ may represent the average number of CPU cycles used to compute one observation point in the object sensing data at (or by) the selected device n′. The computational demand may depend on the total number of observation points in all selected instances of the object sensing data from the device n that maintains (and provides) such an instance, expressed as: Because, in an embodiment, each device n′ may be selected to perform (or support) more than one object classification task, the computational requirements μ for all object classification tasks to be performed at the selected device n′ are n′ It can be expressed as: Denote the computation time of all object classification tasks performed at the selected device n′ as t n′ , which can be expressed as: Among them, f n′ Represents the amount of computing resources available on the selected device n′ (in cycles / second).

[0089] For example, the communication time may depend on the size (e.g., in bits) of the object sensing data being communicated. The total size of all instances of object sensing data to be transmitted from each corresponding device n to the selected device n′ to perform all object (corresponding) classification tasks at or by device n′ may be represented as ρ n,n′ , which can be expressed as: in, Represents the object sensing data size (in bits) of one observation point in the object sensing data. For example, vehicle-to-everything (V2X) transmission based on orthogonal frequency division multiplexing (OFDM) can be used for communication between devices. The average transmission rate on the communication link from the corresponding device n to the selected device n′ is denoted as R n,n′ , which can be expressed as: Where B represents the total bandwidth of the link spectrum shared among all devices; P n Indicates the uplink transmission power of the corresponding device n; h n,n′ represents the channel fading coefficient from the corresponding device n to the selected device n′; d n,n′ represents the distance between the corresponding device n and the selected device n′; γ is the path loss index; σ 2 represents the noise power. For example, for simplicity, assume that the distance d is constant during the perception task n,n′ , considering the low latency requirement of the sensing task, the transmission time of all sensing data from the corresponding device n to the selected device n′ is denoted as t n,n′ , which can be expressed as:

[0090] However, in other embodiments, this constant distance d may not be made. n,n′ assumption.

[0091] The combined (or total) computation time and communication time for all object classification tasks can be expressed as: Where T is a predetermined combined (or total) time threshold. Here, for simplicity, it can be assumed that all corresponding devices n selected to provide object sensing data to other selected devices n' start data transmission at the same time. For simplicity, it can also be assumed that once the selected device n' receives all corresponding instances of object sensing data for all associated object classification tasks, the calculation can be performed at the selected device n'. This scenario leads to the above combined expression of calculation and communication time. Other combined calculation and communication time expressions are also possible.

[0092] By using computation time, communication time, or a combination thereof as a factor (e.g., in constraints or objective functions) in decisions (e.g., data selection, task placement, data parsing, and resource allocation decisions), the controller can facilitate performing object classification in a timely manner, which is important for vehicle control in dynamic environments.

[0093] Although a specific formula is given above for given assumptions, this is provided only as an example and the formula may be varied in many ways, which will be readily understood by those skilled in the art.

[0094] In one embodiment, the controller may allocate resources to minimize or limit the total resource consumption cost o. The total resource consumption cost may include: ′ Device n selected (or specified) for support ′ The computational resources used by all associated object classification tasks at n′ The weighted or unweighted sum of , and the communication resources β to be used for communication between each designated pair of devices (i.e., device n and device n′) in order to support all (associated) object classification tasks n,n′ The total resource consumption cost can be expressed as: Here, f n′ , ω and B are predetermined parameters. Other cost or objective functions may also be used.

[0095] In various embodiments, and more generally, the controller can make decisions to minimize (globally or locally) or limit the cost function, or equivalently maximize (globally or locally) or provide a sufficiently high objective function. Decisions can also be made so that one or more hard constraints or soft constraints are satisfied. Among them, the cost function, the objective function can reflect a combination of one or more factors such as the time spent performing object classification, object classification accuracy, resource usage, security, error probability, etc. Constraints can also reflect a combination of one or more factors of the same type. Therefore, the controller can solve or approximately solve the optimization problem. The decision variables of the optimization problem can represent resource allocation decisions, task delivery decisions, data selection decisions, data parsing decisions, etc., or a combination thereof. For example, the decision variable can be a data selection decision. As another example, the decision variable can be a task delivery decision. As another example, the decision variable can be a resource allocation decision. As another example, the decision variable can be a data parsing decision. As another example, the decision variable can be any two, three or four of a resource allocation decision, a task delivery decision, a data selection decision and a data parsing decision. Various methods for solving or approximately solving the optimization problem can be used, as will be readily understood by those skilled in the art.

[0096] In an embodiment, device n selected at device n to support the object classification task can perform the object classification task based on instructions provided by the controller (e.g., corresponding data selection, data parsing, indications of task placement decisions), allocations provided by the controller (e.g., allocations of computing resources, communication resources), or both.

[0097] In one embodiment, a method for solving or approximately solving an optimization problem may include an external module and an internal module.

[0098] The external module can provide the internal module with candidate technical solutions determined using a genetic algorithm, which include jointly determined data selection, subtask placement, and (optional) data parsing decisions that are feasible at least in terms of accuracy and topological constraints (e.g., a single computing node placement constraint for each object classification task, a half-duplex communication constraint for each AV). The internal module can check the feasibility of the candidate solution under the constraint of a predetermined total (computation and communication) time threshold, and optimize the resource allocation (decision) with the minimum total resource consumption cost.

[0099] Given a candidate technical solution (candidate s, e, g decision), the internal module can determine a resource allocation decision that minimizes the total resource consumption cost o by determining the (total) computation and communication time while satisfying the total time threshold T given the available (or candidate) computation and communication resources.

[0100] In an embodiment, the controller may determine, based at least in part on the context information, an indication of one or more instances of object sensing data to be used in each of the one or more object classification tasks. For example, the context information may be encoded as a cost or objective function, or a constraint, or both, of an optimization problem.

[0101] In an embodiment, the context information may include a network topology of at least those devices participating in or supporting the various methods and systems disclosed herein. The device that obtains the corresponding instance of the object sensing data of the object in the region of interest may be moving or stationary at a corresponding speed in its corresponding direction. The device may move to a location closer to or farther away from the region of interest. Therefore, the network topology may change over time. The communication between the device and the network may be interrupted, for example, due to an obstacle affecting the communication, or due to a failure of the internal communication system of the device.

[0102] In an embodiment, the context information may include resource availability. Resource availability may include information indicating the computing and communication resources available at each device. Resource availability may include information indicating the number of devices available to provide object sensing data for objects in the region of interest. It should be noted that the information indicating the number of devices available to provide object sensing data for objects in the region of interest may be (implicitly or explicitly) included in the object sensing data quality indicator, and in this case, the information may be omitted from the additional resource availability context information. The resource availability information may be used by the controller when allocating resources used by each object classification task in the associated one or more object classification tasks.

[0103] In an embodiment, the context information may include quality of service (QoS) requirement information. Network and device QoS requirements may be preset (eg, predetermined, fixed), and thus may be substantially constant over time.

[0104] In an embodiment, the context information may be obtained by the controller, or by another network element configured to obtain the context information and provide it to the controller, or by both.

[0105] In an embodiment, some context information may be obtained by the controller substantially continuously, for example, if a particular instance of context information may vary irregularly over time, such as usage and possible congestion of local communication resources between devices (e.g., due to system maintenance, updates, or priority communications), or associated delays or limitations, etc. Some context information may be substantially constant over a period of time, such as the QoS requirements discussed above, and thus may be obtained periodically or updated as needed, for example, when a new device enters or approaches an area of ​​interest.

[0106] Figure 6Components of a system for classifying objects provided by an embodiment of the present disclosure are shown. System 500 includes multiple devices 100, and the multiple devices 100 include a target device 110 and at least one other device 120, which can be another AV or stationary device, such as an RSU. Each device in the multiple devices 100 can obtain corresponding raw sensing data 150 of the area of ​​interest. Each device can obtain corresponding abstract sensing data 160 from its raw sensing data 150, as described elsewhere in this document. Each device can transmit its corresponding abstract sensing data 160 to a boundary area module 200. The boundary area module 200 can fuse or combine the corresponding abstract sensing data 160 received from all devices in the multiple devices 100 to determine the boundary area parameters 210 of each object in the area of ​​interest, and the boundary area parameters include the dimensions 220 of the corresponding boundary area that contains or surrounds the object in the area of ​​interest.

[0107] like Figure 6 As further shown in FIG. 1 , the boundary region module 200 sends the boundary region dimension 220 to the controller 300 and transmits the estimated boundary region parameter 210 is sent to each device 100. It should be noted that the boundary area dimension is all estimated boundary area parameters Each device n 100 uses the received boundary area parameters 210 to obtain or extract corresponding object sensing data of at least one object m. 170. Each device 100 also determines the corresponding object sensing data The corresponding object sensing data quality indicator of 170 180 , and sends the corresponding object sensing data quality indicator 180 to the controller 300 .

[0108] The controller 300 may be configured to determine a corresponding instance of the object sensed data 170 to be used in the classification task for each object in the region of interest as part of a collaborative sensing decision 310 (i.e., a data selection decision portion of the overall decision 310). Additionally or alternatively, the controller may be configured to determine other aspects of the overall decision 310, such as a task placement decision, a data parsing decision, a resource allocation decision, or a combination thereof. The controller 300 determines the collaborative sensing decision 310 based on at least the corresponding object sensed data quality indicator 180, the boundary region dimension 220, and the context information 250 discussed elsewhere herein. The controller 300 may receive the context information 250 from a network (not shown), each device 100, another device (not shown) that provides some or all of the context information 250 to the controller, or a combination thereof.

[0109] The controller 300 may provide an indication of data selection 350 to some or all of the devices 100 based on the corresponding data selection decision of the controller, as discussed elsewhere herein. The controller 300 may provide an indication of (optional) data parsing 360 to some or all of the devices 100 based on the corresponding data parsing decision of the controller, as discussed elsewhere herein. The controller 300 may provide an indication of classification task placement 370 to some or all of the devices 100 based on the corresponding task placement decision of the controller, as discussed elsewhere herein. The controller 300 may provide an indication of resource allocation 380 based on the corresponding resource allocation (decision) of the object classification task determined by the controller, as discussed elsewhere herein. The controller 300 may transmit the indication of resource allocation 380 to some or all of the devices 100 (not shown).

[0110] Figure 7 The mechanism of collaborative perception provided by the embodiment of the present disclosure is shown. The collaborative perception decision 310 includes receiving information including boundary area dimension 220, object sensing data quality indicator 180 and context information 250 by the controller. Then, this information is used to determine one or more decisions for performing collaborative perception. These collaborative perception decisions may include one, some or all of data selection decision 350, data parsing decision 360, classification task delivery decision 370 and resource allocation decision 380. The data selection decision 350 indicates that the selected device will provide the corresponding instance of its object sensing data for the object classification task. The data parsing decision 360 indicates the parsing of the object sensing data for the object classification task. The task delivery decision 370 indicates the device selected for each object classification task. The resource allocation decision 380 indicates the computing resources, communication resources or both that will be used to support the classification task. This joint determination can be based on the following: minimize or limit the total resource consumption 385 associated with the object classification task, while setting the classification accuracy 390 to be at or above the predetermined accuracy threshold 391, and setting the total time 395 associated with the completion of all object classification tasks to be at or below the predetermined total time threshold 396. The joint determination may be performed by solving or approximately solving a constrained or unconstrained optimization problem.

[0111] Figure 8 Schematic diagram of an electronic device 1000 provided for different embodiments of the present disclosure, the electronic device 1000 can perform any or all operations of the above methods and features explicitly or implicitly described herein. For example, a computer equipped with network functions can be configured as the electronic device 1000. Such an electronic device can be used as a part of one or more of a controller, an edge server, a processing device, a border area module, an AV, an RSU, etc.

[0112] As shown, the device includes a processor 1010, such as a central processing unit (CPU) or a special processor, such as a graphics processing unit (GPU) or other such processor unit, a memory 1020, a non-transient mass storage 1030, an I / O interface 1040, a network interface 1050 and a transceiver 1060, all of which are coupled by bidirectional bus 1070 communication. According to certain embodiments, any or all of the elements depicted can be used, or only a subset of the elements can be used. In addition, the device 1000 can include multiple instances of certain elements, such as multiple processors, memories or transceivers. In addition, the elements of the hardware device can be directly coupled to other elements without bidirectional bus. As a supplement or alternative to processors and memories, other electronic components such as integrated circuits can be used to perform required logical operations.

[0113] The memory 1020 may include any type of non-transient memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or a combination thereof. The mass storage unit 1030 may include any type of non-transient storage device, such as a solid-state drive, a hard disk drive, a disk drive, an optical drive, a USB drive, or any computer program product for storing data and machine executable program code. According to some embodiments, the memory 1020 or the mass storage 1030 may record thereon statements and instructions executable by the processor 1010 for performing any of the above-mentioned method operations.

[0114] It should be understood that although specific embodiments of the technology have been described herein for illustrative purposes, various modifications may be made without departing from the scope of the technology. The specification and drawings are to be considered merely as an illustration of the invention as defined by the appended claims and are considered to cover any and all modifications, variations, combinations or equivalents falling within the scope of the invention. Specifically, the following is within the scope of the technology: providing a computer program product or program element, or a program storage element or memory device (e.g., magnetic or optical wire, tape or disk, etc.) for storing machine-readable signals, controlling the operation of a computer according to the method of the technology, and / or constructing some or all of its components according to the system of the technology.

[0115] The actions associated with the methods described herein may be implemented as coded instructions in a computer program product. In other words, a computer program product is a computer-readable medium in which software codes are recorded to perform the method when the computer program product is loaded into a memory and executed on a microprocessor of a wireless communication device.

[0116] Furthermore, each operation of the method may be executed on any computing device, such as a personal computer, server, PDA, etc., according to one or more or a portion of one or more program elements, modules or objects generated from any programming language such as C++, Java, etc. Furthermore, each operation or a file or object that implements each of the operations, etc. may be executed by dedicated hardware or a circuit module designed for this purpose.

[0117] Through the description of the above embodiments, the present invention can be implemented only by hardware, or by software and necessary general hardware platforms. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transient storage medium, and the non-volatile or non-transient storage medium can be a compact disk read-only memory (CD-ROM), a USB flash drive, or a mobile hard disk. The software product includes several instructions that enable a computer device (a personal computer, a server, or a network device) to execute the method provided in the embodiment of the present invention. For example, such execution may correspond to a simulation of a logical operation as described herein. The software product may additionally or alternatively include several instructions that enable a computer device to perform operations of configuring or programming a digital logic device provided in an embodiment of the present invention.

[0118] Although the present invention has been described with reference to specific features and embodiments of the present invention, it is apparent that various modifications and combinations of the present invention may be made without departing from the present invention. The specification and drawings are to be regarded only as illustrations of the present invention as defined by the appended claims and are considered to cover any and all modifications, variations, combinations or equivalents falling within the scope of the present invention.

Claims

1. A method comprising: a controller receiving object sensing data quality indicators determined by a plurality of devices including a target device, each object sensing data quality indicator indicating a quality of a corresponding instance of object sensing data, wherein the corresponding instance of the object sensing data is maintained by a corresponding device of the plurality of devices and indicates a corresponding object, the corresponding object belonging to a set of objects in a region of interest of at least the target device; The controller provides, to at least one of the plurality of devices, an indication of one or more of the instances of the object sensing data to be used in each of one or more object classification tasks, each of the object classification tasks being used to classify a corresponding object in the set of objects.

2. The method according to claim 1, wherein: The providing of an indication of one or more instances of the object sensing data includes providing a data selection indication to a specific device among the at least one device among the multiple devices, the data selection indication indicating that an instance of the object sensing data maintained by the specific device and indicating a specific object is to be used in a specific corresponding object classification task in the object classification task, wherein the specific object belongs to the object set.

3. The method according to claim 2, wherein: The particular device of the at least one of the plurality of devices is selected by the controller based at least in part on an object sensing data quality indicator that indicates a quality of a corresponding instance of object sensing data maintained by the particular device.

4. The method according to any one of claims 1 to 3, further comprising: The controller provides one or more task placement indications to at least one of the multiple devices, each task placement indication indicating that a specified one of the object classification tasks will be performed at a specified one of the multiple devices, wherein the task placement indication is determined together with the following indications: the indication of one or more instances of the instance of the object sensing data to be used in each of the one or more object classification tasks.

5. The method according to claim 4, wherein: At least one of the task placement indications comprises the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

6. The method according to claim 4 or 5, further comprising: The controller allocates resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by the designated device of the plurality of devices to support associated one or more of the object classification tasks; a communication resource, the communication resource being used for communicating between the designated device in the plurality of devices and another designated device in the plurality of devices to support the associated one or more object classification tasks in the object classification tasks, Wherein the controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

7. The method according to claim 6, wherein: The resources are allocated to at least partially satisfy: a combination of the following times will be at or below a predetermined threshold: the computation time to complete each of the associated one or more object classification tasks in the object classification task, and the communication time for communicating between the specified device in the multiple devices and other specified devices in the multiple devices to support each of the associated one or more object classification tasks in the object classification task.

8. The method according to claim 6 or 7, further comprising: The controller transmits an indication of the allocated resources to one or more devices of the plurality of devices, and the plurality of devices use the allocated resources when performing the object classification task and avoid using more resources than the allocated resources when performing the object classification task.

9. The method according to any one of claims 4 to 8, further comprising: The designated device among the multiple devices executes the designated object classification task among the object classification tasks according to the one or more task delivery instructions.

10. The method according to any one of claims 1 to 9, further comprising: The controller provides one or more data parsing indications to at least one of the plurality of devices, each data parsing indication indicating that an associated instance of the instance of the object sensing data to be used in each of one or more object classification tasks is to be downsampled to a specified degree prior to use in the associated object classification task, the use including transmitting the object sensing data between members of the plurality of devices when necessary.

11. The method according to claim 10, further comprising: The at least one device of the plurality of devices downsamples the associated instances of the instances of the object sensing data by the specified degree based on the one or more data parsing instructions.

12. The method according to any one of claims 1 to 6 and 10, further comprising: The controller allocates resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by a designated device of the plurality of devices to support an associated one of the object classification tasks; a communication resource to be used for communicating between a designated pair of devices in the plurality of devices to support the associated one of the object classification tasks, Wherein the controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

13. The method according to claim 12, wherein: The controller performs the resource allocation to: Minimize or limit the total resource consumption cost, the total resource consumption cost comprising: a weighted sum or an unweighted sum of the computing resources to be used by each designated device among the multiple devices to support all associated object classification tasks, and a weighted sum or an unweighted sum of the communication resources to be used for communication between each designated pair of devices among the multiple devices to support all object classification tasks.

14. The method according to any one of claims 1 to 13, further comprising: At least one of the multiple devices performs the object classification task based on an indication, an assignment, or both an indication and an assignment provided by the controller, wherein the indication includes an indication of one or more instances of instances of the object sensing data to be used in each of the one or more object classification tasks.

15. The method according to any one of claims 1 to 14, further comprising: Determining instances of the object sensing data, the determining comprising: for each instance of the object sensing data: A bounding region module for determining a bounding region of an object indicated by an instance of the object sensing data sends bounding region parameters of the bounding region to each of the plurality of devices; Each device of the plurality of devices determines a corresponding instance of the object sensing data from raw sensing data obtained by the device of the plurality of devices using the boundary region parameters.

16. The method according to claim 15, wherein: The boundary region module is located at the controller or at the at least one device among the plurality of devices.

17. The method according to claim 15 or 16, further comprising: providing the indication of the one or more instances of the object sensing data to satisfy: an estimated classification accuracy of each of the object classification tasks is at or above a predetermined classification accuracy threshold, the estimated classification accuracy being determined by the controller based at least in part on a corresponding object sensing data quality indicator for each object classification task in conjunction with dimensions of a boundary region of the object, the dimensions being received by the controller from the boundary region module.

18. The method according to claim 17, wherein: An estimated classification accuracy for each of the object classification tasks is indicated by the controller to the device.

19. The method according to any one of claims 1 to 18, wherein: The region of interest is a region of interest of a plurality of target devices including the target device.

20. The method according to claim 19, wherein: The region of interest is a union of sub-regions, and each sub-region is a region of interest of a corresponding target device among the multiple target devices.

21. The method according to claim 19 or 20, wherein: The plurality of devices include the plurality of target devices.

22. The method according to any one of claims 1 to 21, wherein: At least the following indications are determined based on context information: the indications of one or more of the instances of the object sensing data, the context information comprising one or more of: network topology context information, resource availability context information, and quality of service requirement context information.

23. The method according to any one of claims 1 to 22, further comprising: The at least one device among the plurality of devices performs the one or more object classification tasks.

24. A computer program product comprising instructions, wherein: When the instructions are executed by a computer, the computer is caused to implement the method according to any one of claims 1 to 19.

25. A system comprising: a plurality of devices, the plurality of devices including a target device and configured to determine object sensing data quality indicators, each object sensing data quality indicator indicating a quality of a respective instance of object sensing data, wherein the respective instance of object sensing data is maintained by a respective device of the plurality of devices and indicates a respective object, the respective object belonging to a set of objects in a region of interest of at least the target device; a controller for receiving the object sensing data quality indicator and providing, to at least one of the plurality of devices, an indication of one or more instances of the object sensing data to be used in each of one or more object classification tasks, each of the object classification tasks being used to classify a corresponding object in the set of objects, Each of the plurality of devices and the controller communicate with each other.

26. The system of claim 25, wherein: The controller providing the indication of the one or more instances of the instance of the object sensing data includes: providing a data selection indication to a specific device among the at least one device among the multiple devices, the data selection indication indicating that: the instance of the object sensing data maintained by the specific device and indicating a specific object is to be used in a specific corresponding object classification task in the object classification task, wherein the specific object belongs to the object set.

27. The system of claim 26, wherein: The particular device of the at least one of the plurality of devices is selected by the controller based at least in part on an object sensing data quality indicator that indicates a quality of a corresponding instance of object sensing data maintained by the particular device.

28. A system according to any one of claims 25 to 27, wherein: The controller is also used to provide one or more task placement indications to at least one device among the multiple devices, each task placement indication indicating that a specified object classification task among the object classification tasks will be performed at a specified device among the multiple devices, wherein the task placement indication is determined together with the following indications: the indication of one or more instances of the instance of the object sensing data to be used in each object classification task among the one or more object classification tasks.

29. The system of claim 28, wherein: At least one of the task placement indications comprises the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

30. The system of claim 28 or 29, wherein: The controller is further configured to allocate resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by the designated device of the plurality of devices to support associated one or more of the object classification tasks; a communication resource, the communication resource being used for communicating between the designated device in the plurality of devices and another designated device in the plurality of devices to support the associated one or more object classification tasks in the object classification tasks, Wherein the controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

31. The system of claim 30, wherein: The resources are allocated to at least partially satisfy: a combination of the following times will be at or below a predetermined threshold: the computation time to complete each of the associated one or more object classification tasks in the object classification task, and the communication time for communicating between the specified device in the multiple devices and other specified devices in the multiple devices to support each of the associated one or more object classification tasks in the object classification task.

32. A system according to claim 30 or 31, wherein: The controller is also used to transmit an indication of the allocated resources to one or more devices among the multiple devices, and the multiple devices use the allocated resources when performing the object classification task and avoid using more resources than the allocated resources when performing the object classification task.

33. A system according to any one of claims 28 to 32, wherein: The designated device among the multiple devices is further configured to execute the designated object classification task among the object classification tasks according to the one or more task delivery instructions.

34. A system according to any one of claims 25 to 33, wherein: The controller is also used to provide one or more data parsing indications to at least one of the multiple devices, each data parsing indication indicating that an associated instance of the instance of the object sensing data to be used in each of one or more object classification tasks will be downsampled to a specified degree before being used in the associated object classification task, wherein the use includes transmitting the object sensing data between members of the multiple devices when necessary.

35. The system of claim 34, wherein: The at least one device among the plurality of devices is further configured to downsample the associated instances of the instances of the object sensing data by the specified degree according to the one or more data parsing indications.

36. A system according to any one of claims 25 to 29 and 34, wherein: The controller is further configured to allocate resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by a designated device of the plurality of devices to support an associated one of the object classification tasks; a communication resource to be used for communicating between a designated pair of devices in the plurality of devices to support the associated one of the object classification tasks, Wherein the controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

37. The system of claim 36, wherein: The controller performs the resource allocation to: Minimize or limit the total resource consumption cost, the total resource consumption cost comprising: a weighted sum or an unweighted sum of the computing resources to be used by each designated device among the multiple devices to support all associated object classification tasks, and a weighted sum or an unweighted sum of the communication resources to be used for communication between each designated pair of devices among the multiple devices to support all object classification tasks.

38. A system according to any one of claims 25 to 37, wherein: The at least one device among the multiple devices is used to perform the object classification task according to the instructions, assignments, or both instructions and assignments provided by the controller, wherein the instructions include the indications of one or more instances of instances of the object sensing data to be used in each of the one or more object classification tasks.

39. A system according to any one of claims 25 to 38, wherein: Examples of determining the object sensing data include: A bounding region module for determining a bounding region of an object indicated by an instance of the object sensing data sends bounding region parameters of the bounding region to each of the plurality of devices; Each device of the plurality of devices determines a corresponding instance of the object sensing data from raw sensing data obtained by the device of the plurality of devices using the boundary region parameters.

40. The system of claim 39, wherein: The boundary region module is located at the controller or at the at least one device among the plurality of devices.

41. The system of claim 39 or 40, further configured to provide the indication of the one or more instances of the object sensing data to satisfy: an estimated classification accuracy of each of the object classification tasks is at or above a predetermined classification accuracy threshold, the estimated classification accuracy being determined by the controller in combination with dimensions of a boundary region of the object, at least in part based on a corresponding object sensing data quality indicator for each object classification task, the dimensions being received by the controller from the boundary region module.

42. The system of claim 41, wherein: An estimated classification accuracy for each of the object classification tasks is indicated by the controller to the device.

43. A system according to any one of claims 25 to 42, wherein: The region of interest is a region of interest of a plurality of target devices including the target device.

44. The system of claim 43, wherein: The region of interest is a union of sub-regions, and each sub-region is a region of interest of a corresponding target device among the multiple target devices.

45. A system according to claim 43 or 44, wherein: The plurality of devices include the plurality of target devices.

46. ​​A system according to any one of claims 25 to 45, wherein: At least the following indications are determined based on context information: the indications of one or more of the instances of the object sensing data, the context information comprising one or more of: network topology context information, resource availability context information, and quality of service requirement context information.

47. A system according to any one of claims 25 to 46, wherein: The at least one device among the plurality of devices is further configured to perform the one or more object classification tasks.

48. A controller for: receiving object sensing data quality indicators determined by a plurality of devices including a target device, each object sensing data quality indicator indicating a quality of a corresponding instance of object sensing data, wherein The respective instance of the object sensing data is maintained by a respective device of the plurality of devices and indicates a respective object, the respective object belonging to a set of objects in a region of interest of at least the target device; An indication of one or more of the instances of the object sensing data to be used in each of one or more object classification tasks is provided to at least one device of the plurality of devices, each of the object classification tasks being used to classify a corresponding object in the set of objects.

49. A controller according to claim 48, wherein: The controller providing the indication of the one or more instances of the instance of the object sensing data includes: providing a data selection indication to a specific device among the at least one device among the multiple devices, the data selection indication indicating that: the instance of the object sensing data maintained by the specific device and indicating a specific object is to be used in a specific corresponding object classification task in the object classification task, wherein the specific object belongs to the object set.

50. The controller of claim 48, further configured to select the particular device of the at least one of the plurality of devices based at least in part on an object sensing data quality indicator indicating a quality of a corresponding instance of object sensing data maintained by the particular device.

51. The controller according to any one of claims 48 to 50 is further configured to provide one or more task delivery instructions to the at least one device among the plurality of devices, each task delivery instruction indicating that a specified object classification task among the object classification tasks is to be executed at a specified device among the plurality of devices, wherein The task placement indication is determined together with the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

52. A controller according to claim 51, wherein: At least one of the task placement indications comprises the indication of one or more of the instances of the object sensing data to be used in each of the one or more object classification tasks.

53. The controller according to claim 51 or 52, further configured to allocate resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by the designated device of the plurality of devices to support associated one or more of the object classification tasks; a communication resource, the communication resource being used for communicating between the designated device in the plurality of devices and another designated device in the plurality of devices to support the associated one or more object classification tasks in the object classification tasks, in, The controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of one or more object classification tasks.

54. A controller according to claim 53, wherein: The resources are allocated to at least partially satisfy: a combination of the following times will be at or below a predetermined threshold: the computation time to complete each of the associated one or more object classification tasks in the object classification task, and the communication time for communicating between the designated device in at least one of the multiple devices and other designated devices in the multiple devices to support each of the associated one or more object classification tasks in the object classification task.

55. The controller according to claim 53 or 54 is also used to transmit an indication of the allocated resources to one or more devices among the multiple devices, and the multiple devices use the allocated resources when performing the object classification task and avoid using more resources than the allocated resources when performing the object classification task.

56. A controller according to any one of claims 51 to 55, wherein: The designated device among the multiple devices executes the designated object classification task among the object classification tasks according to the one or more task delivery instructions.

57. A controller according to any one of claims 48 to 56, further configured to provide one or more data parsing indications to at least one of the plurality of devices, each data parsing indication indicating that an associated instance of the instance of the object sensing data to be used in each of one or more object classification tasks will be downsampled to a specified degree before being used in the associated object classification task, the use including transmitting the object sensing data between members of the plurality of devices when necessary.

58. A controller according to claim 57, wherein: The at least one device among the plurality of devices is configured to downsample the associated instances of the instances of the object sensing data by the specified degree according to the one or more data parsing indications.

59. The controller according to any one of claims 48 to 52 and 57, further configured to allocate resources for each of the one or more object classification tasks, the resources comprising one or both of the following: computing resources to be used by a designated device of the plurality of devices to support an associated one of the object classification tasks; a communication resource to be used for communicating between a designated pair of devices in the plurality of devices to support the associated one of the object classification tasks, in, The controller performs the resource allocation in conjunction with determining the indication of one or more of the instances of the object sensing data to be used in each of one or more object classification tasks.

60. The controller of claim 59, wherein: The controller performs the resource allocation to: Minimize or limit the total resource consumption cost, the total resource consumption cost comprising: a weighted sum or an unweighted sum of the computing resources to be used by each designated device among the multiple devices to support all associated object classification tasks, and a weighted sum or an unweighted sum of the communication resources to be used for communication between each designated pair of devices among the multiple devices to support all object classification tasks.

61. A controller according to any one of claims 48 to 60, wherein: At least one of the multiple devices performs the object classification task based on an indication, an assignment, or both an indication and an assignment provided by the controller, wherein the indication includes an indication of one or more instances of instances of the object sensing data to be used in each of the one or more object classification tasks.

62. A controller according to any one of claims 48 to 61, wherein: Also included or operatively coupled to a boundary area module located at the controller or at another device, the boundary area module being configured to: determining a bounding region of an object, wherein the object is indicated by an instance of the object sensing data; sending a boundary area parameter of the boundary area to each of the plurality of devices; Each of the plurality of devices uses the boundary region parameters to determine a corresponding instance of the object sensing data from raw sensing data obtained by the device of the plurality of devices.

63. The controller according to claim 62, further configured to: receiving, from the bounding area module, dimensions of a bounding area of ​​the object; determining an estimated classification accuracy for each of the object classification tasks based at least in part on a corresponding object sensing data quality indicator for each of the object classification tasks in conjunction with the dimensions, in, The indications of the one or more instances of the object sensing data are provided to at least partially satisfy that an estimated classification accuracy for each of the object classification tasks is at or above a predetermined classification accuracy threshold.

64. The controller of claim 63, further configured to indicate to the device an estimated classification accuracy for each of the object classification tasks.

65. A controller according to any one of claims 48 to 63, wherein: The region of interest is a region of interest of a plurality of target devices including the target device.

66. A controller according to claim 65, wherein: The region of interest is a union of sub-regions, and each sub-region is a region of interest of a corresponding target device among the multiple target devices.

67. A controller according to claim 65 or 66, wherein: The plurality of devices include the plurality of target devices.

68. A controller according to any one of claims 48 to 67, wherein: At least the following indications are determined based on context information: the indications of one or more of the instances of the object sensing data, the context information comprising one or more of: network topology context information, resource availability context information, and quality of service requirement context information.

69. A controller according to any one of claims 48 to 68, wherein: The at least one device among the plurality of devices performs the one or more object classification tasks.

Images