Perceptual correlation process

Abstract

The present disclosure provides a computer-implemented method for perceiving a related process, the method comprising: (i) receiving, by a processing circuit of a vehicle, scene information about a scene in which the vehicle faces; wherein the scene information comprises environment information about the environment of the vehicle; (ii) identifying, by the processing circuitry, the scene using the received scene information; (iii) determining, based on the identified scene, a resource operation parameter that conforms to the identified scene and is related to the operation of the perceptual related process; and (iv) making the resource operation parameters available to the operation of the perceptual relevant process.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly, to a method, a non-transitory computer-readable medium, and a system for performing a perception process. Background Art

[0002] Assisted and autonomous driving systems are known in the art. In such systems, a computer-implemented system controls (at least to some extent) some or all of a vehicle's driving functions, such as speed, telemetry, braking, etc. The vehicle is typically equipped with one or more sensors to provide the system with current information about the driving environment. The driving system typically uses this current information about the driving environment to determine how to steer on the road.

[0003] One of the main tasks associated with driving is classification.

[0004] Therefore, there is an increasing need to provide efficient classification systems and methods. Summary of the Invention

[0005] The present disclosure provides a method, a non-transitory computer-readable storage medium, and a computer-implemented system for performing a perception process.

[0006] In a first aspect of the present disclosure, a computer-implemented method for a perception-related process is provided. The method includes: receiving, by processing circuitry of a vehicle, scene information about a scene encountered by the vehicle, wherein the scene information includes environmental information about an environment of the vehicle; identifying, by the processing circuitry, a scene using the received scene information; determining, based on the identified scene, resource operating parameters that are consistent with the identified scene and relevant to the operation of the perception-related process; and making the resource operating parameters available in the operation of the perception-related process.

[0007] In another aspect of the present disclosure, a non-transitory computer-readable medium for a perception-related process stores instructions that, once executed by a computerized system, cause the computerized system to perform the following operations: receiving, by a processing circuit of a vehicle, scene information about a scene faced by the vehicle, wherein the scene information includes environmental information about the environment of the vehicle; identifying, by the processing circuit, the scene using the received scene information; determining, based on the identified scene, resource operation parameters that are consistent with the identified scene and are related to the operation of the perception-related process; and making the resource operation parameters available in the operation of the perception-related process.

[0008] In another aspect of the present disclosure, a computerized system for a perception-related process includes: a memory unit configured to store scene information about a scene faced by a vehicle; wherein the scene information includes environmental information about the environment of the vehicle; and a processing circuit configured to: use the received scene information to identify a scene; based on the identified scene, determine resource operation parameters that are consistent with the identified scene and related to the operation of the perception-related process; and make the resource operation parameters available in the operation of the perception-related process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The embodiments of the present disclosure will be more fully understood and appreciated through the following detailed description in conjunction with the accompanying drawings, in which:

[0010] Figure 1A A block diagram illustrating an example of a system according to an embodiment of the present disclosure;

[0011] Figure 1B A block diagram illustrating an example of a system according to an embodiment of the present disclosure;

[0012] Figure 1C A block diagram illustrating an example of a system according to an embodiment of the present disclosure;

[0013] Figure 2A A flowchart illustrating an example of a method according to an embodiment of the present disclosure;

[0014] Figure 2B A flowchart illustrating an example of a method according to an embodiment of the present disclosure;

[0015] Figure 3 is a block diagram of an example of a perception-related system according to an embodiment of the present disclosure;

[0016] Figure 4 is an example of an image according to an embodiment of the present disclosure; and

[0017] Figure 5 is an example of an image according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The various figures illustrate examples of units and / or software and / or information items and / or steps and / or components. These examples are provided for simplicity of explanation. At least one of the units and / or software and / or information items and / or steps and / or components may be optional or mandatory.

[0019] According to one embodiment, a method, system, and non-transitory computer-readable medium are provided for scenario-based perception-related processes, particularly for controlling the manner in which the perception-related processes are performed, thereby providing optimized or sub-optimal resource allocation and / or usage during driving.

[0020] According to one embodiment, the method comprises:

[0021] a) Receive, by a processing circuit of a vehicle, scene information of a scene faced by the vehicle; wherein the scene information includes environmental information about the environment of the vehicle.

[0022] b) The processing circuit uses the received scene information to identify the scene.

[0023] c) Based on the identified scenario, determining resource operating parameters that are consistent with the identified scenario and are related to the operation of the perception-related process.

[0024] d) Make resource operation parameters available in the operation of sensing related processes.

[0025] Figure 1A 、 Figure 1B and Figure 1C An example of a vehicle 100 , a network 123 , and a remote computerized system 134 is shown.

[0026] exist Figure 1A , vehicle 100 is shown as including a sensing system 110 , a communication system 130 , one or more memory and / or storage units 120 , a control unit 125 ′, a network 132 in communication with a remote computerized system 134 .

[0027] One or more memories and / or storage units 120 are shown storing information 191, metadata 192, software 193, and operating system 194. Information 191, metadata 192, software 193, and operating system 194 are required to perform one or more methods explained in the specification.

[0028] exist Figure 1B and Figure 1C In the embodiment of the present invention, the control unit 125' is replaced by different components, such as an advanced driver assistance system (ADAS) control unit 123, an autonomous driving control unit 122, a vehicle computer 121, and a controller 125. Note that only some or all of these components may be included in the vehicle.

[0029] Figure 1B Examples of one or more types of information 191 and metadata 192 stored in one or more memories and / or storage units 120 are also provided.

[0030] Figure 1C Examples of one or more types of software 193 stored in one or more memories and / or storage units 120 are also provided.

[0031] The vehicle 100 includes a sensing system 110, a communication system 130, one or more memory and / or storage units 120, and additional units including an advanced driver assistance system (ADAS) control unit 123, an autonomous driving control unit 122, a vehicle computer 121, a controller 125, and a processing system 124 including a processor 126. The network 123 communicates with the vehicle and a remote computerized system 134, such as a server or cloud computer.

[0032] The communication system 130, one or more memories and / or storage units 120, and the processing system 134 may form a computerized system. The computerized system may include one or more other systems and / or units, such as the sensing system 110 (at least the image signal processor 114), the ADAS control unit 123, the autonomous driving control unit 122, the vehicle computer 121, and the controller 125.

[0033] According to one embodiment, sensing system 110 includes optics 111, a sensing element group 112, readout circuitry 113, and an image signal processor 114. Optics 111 is followed by sensing element group 112, such as a line of sensing elements or an array of sensing elements forming the sensing element group. The sensing element group is followed by readout circuitry 113, which reads the detection signals generated by the sensing element group. Image signal processor 114 is configured to perform initial processing of the detection signals, such as by improving the quality of the detection information, performing noise reduction, and the like. Sensing system 110 is configured to output one or more sensing information units (SIUs).

[0034] The communication system 130 is configured to enable communication between one or more memories and / or storage units 120 and / or the sensing system 110 and / or any additional units and / or a network 132 (in communication with a remote computerized system).

[0035] The controller 125 is configured to control the operation of the sensing system 110 , and / or the one or more memories and / or storage units 120 , and / or one or more additional units (other than the controller).

[0036] The ADAS control unit 123 is configured to control ADAS operations. ADAS operations may include performing autonomous maneuvers (e.g., emergency braking, performing short-term autonomous driving maneuvers—e.g., lane keeping and / or autonomous parking and / or autonomously driving the vehicle for a short period of time—which may range from 0.1 to 2, 3, 4, 5, 6, 7, 8, 9, 10 seconds, etc.), suggesting driving maneuvers to be performed by a human driver, and suggesting a path to be followed by a human driver driving the vehicle.

[0037] The autonomous driving control unit 122 is configured to control the autonomous driving of the autonomous vehicle.

[0038] The vehicle computer 121 is configured to control the operation of the vehicle, in particular the engine, the transmission, and any other vehicle systems or components.

[0039] The processing system 124 may include the processor 126 and one or more other processors and may be configured to perform any of the methods explained in the specification.

[0040] The one or more memories and / or storage units 120 are configured to store firmware and / or software, one or more operating systems, data, and metadata required to perform any of the methods mentioned herein.

[0041] Figure 1B One or more memories and / or storage units 120 are shown as storing:

[0042] a) Scenario information 161, which may include at least one of the following:

[0043] i. Road Setting Information 161-1

[0044] ii. Road User Information 161-2

[0045] iii. Traffic Regulation Information 161-3

[0046] iv. Environmental Conditions Information 161-4

[0047] v. Ego vehicle kinematic information 161-5

[0048] b) Information about different scenarios—e.g., clusters representing scenes 162

[0049] c) Indicators and / or parameters, such as resource operating parameters 163-1, applicable performance indicators 163-2, and applicable trade-off indicators 163-3. Applicable performance indicators 163-2 and / or applicable trade-off indicators 163-3 can be determined in any manner and / or by any entity (end user, one or more suppliers, mechanic) based on previous use of the vehicle, based on history, based on training, based on machine learning, etc.

[0050] Figure 1C One or more types of software are shown, including:

[0051] d) Road User Impact Software 170

[0052] e) Scene identification software 171

[0053] f) One or more perception module software 172

[0054] g) Preprocessing software 173

[0055] h) Resource operation parameter determination software 174

[0056] i) Perception-related processing software 175

[0057] j) Machine Learning Processing Software 176

[0058] k) Additional software 177 that can be used to perform any other functions of the vehicle and / or any other unit shown in FIG. 1

[0059] l) Operating System 178

[0060] m) Change rate software 179

[0061] n) Response software

[0062] o) Narrow AI agent software 165

[0063] The vehicle computer 121 may communicate with an engine control module, a transmission control module, a powertrain control module, and the like.

[0064] The memory and / or storage unit 120 is shown as storing software. Any references to software should be made mutatis mutandis to code and / or firmware and / or instructions and / or commands and so forth.

[0065] Processor 126 includes a plurality of processing circuits 126(1)-126(J), where J is an integer greater than 1. Any reference to one unit or item should be applied mutatis mutandis to multiple units or items. For example, any reference to a processor should be applied mutatis mutandis to multiple processors, and any reference to communication system 130 should be applied mutatis mutandis to multiple communication systems.

[0066] According to one embodiment, the one or more memories and / or storage units 120 include one or more memory units, each of which may include one or more memory banks.

[0067] According to one embodiment, the one or more memories and / or storage units 120 include volatile memory and / or non-volatile memory. The one or more memories and / or storage units 120 may be random access memory (RAM) and / or read-only memory (ROM).

[0068] According to one embodiment, the non-volatile storage unit is a mass storage device that can provide non-volatile storage of computer code, computer-readable instructions, data structures, program modules, and other data for the processor or any other unit of the vehicle. For example, but not limited to, the mass storage device can be a hard disk, a removable magnetic disk, a removable optical disk, a magnetic tape cassette or other magnetic storage device, a flash memory card, a CD-ROM, a digital versatile disk (DVD) or other optical storage, a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0069] Any content may be stored in any portion of or in any type of memory and / or storage.

[0070] According to one embodiment, at least one memory unit stores at least one database - such as any database known in the art - e.g. Access, SQL Server, mySQL, PostgreSQL, etc.

[0071] The various units and / or components communicate with each other using any communication elements and / or protocols. One example of a communication system is shown as 130. Other communication elements may be provided.

[0072] FIG. 1 shows a communication system 130 in communication with various processors and / or units and a network 132 .

[0073] Communication system 130 can include bus.Represent one or more of several possible types of bus structures, including memory bus or memory controller, peripheral bus, accelerated graphics port, and processor or local bus using any of various bus architectures.As an example, such architecture can include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, Accelerated Graphics Port (AGP) bus and Peripheral Component Interconnect (PCI), PCI-Express bus, Personal Computer Memory Card Industry Association (PCMCIA), Universal Serial Bus (USB) etc.The bus specified in this manual and all buses can also be realized by wired or wireless network connection and each subsystem.

[0074] The network 132 is located outside the vehicle and is used for communication between the vehicle and at least one remote computing system. As an example, the remote computing system can be a personal computer, a laptop computer, a portable computer, a server, a router, a network computer, a peer device, or other public network node. The logical connection between the processor and any remote computing system can be implemented via a local area network (LAN) and a general wide area network (WAN). Such network connection can be through a network adapter (which can belong to the communication system 130), which can be implemented in a wired and wireless environment. Such networking environments are conventional and common in offices, enterprise-wide computer networks, intranets, and larger networks such as the Internet.

[0075] It should be noted that at least a portion of the content illustrated as stored in the one or more memories / storage units 120 may be stored external to the vehicle.It should also be noted that the processor may evaluate features generated by multiple detectors.

[0076] According to one embodiment, while using one or more memories / storage units 120 and communication system 130, the processor is configured to:

[0077] a) Receive scene information 161 about the scene faced by the vehicle. The scene information may include at least one of road setting information 161-1, road user information 161-2, traffic regulation information 161-3, adjustment information, environmental condition information 161-4, etc.

[0078] b) Feeding scene information 161 to multiple perception modules. The multiple perception modules can be implemented by executing one or more perception module software 172.

[0079] c) Using the scene information to identify the scene, for example, by executing scene identification software 171 to identify the scene.

[0080] d) Based on the identified scenario, determining resource operating parameters that are consistent with the identified scenario and relevant to operation of the perception-related process, for example, by executing resource operating parameter determination software 174. This determination may be based on one or more parameters and / or indicators, such as applicable performance indicator 163-2 and / or applicable tradeoff indicator 163-3 and / or a rate of change within the environment (which may be determined by executing rate of change software 179) and / or an estimated impact of road users within the environment on vehicle driving (which may be determined by executing road user impact software 170).

[0081] e) responding to the determination (eg, by executing responsive software). Responding may include, for example, (i) making resource operating parameters available in operation of a sensing-related process, and / or (ii) executing a sensing-related process.

[0082] According to one embodiment, one or more of these steps may be performed by one or more machine learning processes, such as by executing machine learning processing software 176 .

[0083] Figure 2A and Figure 2B A flow chart illustrating an example of a method 200 for a perception correlation process is shown.

[0084] According to one embodiment, the method 200 comprises a step 210 of receiving, by a processing circuit of a vehicle, context information about a context faced by the vehicle. The context information comprises environmental information about the environment of the vehicle.

[0085] According to one embodiment, the scenario information further includes at least one of road setting information, road user information, traffic rule information, adjustment information, environmental condition information, etc.

[0086] According to one embodiment, the scene information is multi-dimensional information, for example, a combination of at least two or more of the following information:

[0087] a) Road setting information, e.g., static road features and context, such as lanes, interchanges, merges, cities, and countries.

[0088] b) Road user information, e.g., dynamic players, their attributes and impact on the ego vehicle, e.g., the status of crossing pedestrians, rolling balls.

[0089] c) Traffic regulation information, e.g., temporary or permanent restrictions, such as speed limits, traffic signs.

[0090] d) Environmental condition information, e.g., vision and more characteristics, such as rain, night, wind, tunnel.

[0091] e) Ego vehicle kinematic information, e.g., ego vehicle movement, such as speed, yaw rate, acceleration / deceleration.

[0092] According to one embodiment, the multidimensional information is fed to the plurality of perception modules, and one or more segments of the multidimensional information are fed to the plurality of perception modules or any representation or metadata about at least a portion of the plurality of perception modules.

[0093] According to one embodiment, scene information is fed to a single perception module.

[0094] According to one embodiment, the scene information comprises one or more sensed information units sensed by one or more sensors associated with the vehicle. The sensors associated with the vehicle are selected from vehicle sensors and off-vehicle sensors that monitor the vehicle (for at least a defined period of time).

[0095] According to one embodiment, the scene information is the result of preprocessing one or more sensory information units. Preprocessing can include noise reduction and / or segmentation and / or any preliminary steps that help detect the scene. For example, preprocessing can include detecting objects, defining boundary shapes, etc.

[0096] According to one embodiment, step 210 is followed by step 220 , where the processing circuit uses the received context information to identify the context.

[0097] According to one embodiment, the identifying includes classifying the scene by applying any classification process, such as a machine learning classification process.

[0098] According to one embodiment, step 220 comprises finding a match between a signature of a portion of the scene information and a cluster signature of a cluster associated with the scene.

[0099] According to one embodiment, the identification is based on a training process. According to one embodiment, the training is a self-learning training process comprising:

[0100] a) Collecting unlabeled information. According to one embodiment, collecting includes collecting multi-dimensional information, such as at least two types of information among road configuration information, road user information, traffic regulation information, regulation information, environmental condition information, and vehicle kinematics information.

[0101] b) Perform unsupervised clustering of multi-dimensional information representing different scenarios.

[0102] According to one embodiment, clustering is based on at least some of said dimensions of information - e.g. clustering information obtained when a vehicle is travelling on a highway at a specified speed range - e.g. clustering information obtained when a vehicle is travelling in a built-up area in the rain - e.g. a vehicle is driving on a roundabout at another specified speed range.

[0103] According to one embodiment, step 220 is followed by step 230 of determining resource operating parameters that are consistent with the identified scenario and are relevant to the operation of the perception-related process based on the identified scenario.

[0104] According to one embodiment, the resource operation parameters include at least one of the following:

[0105] a) Resolution of the sensor.

[0106] b) Resolution associated with the processing operation. Different resolutions may be applied to different regions of the sensed information unit (or to different regions of any information generated based at least in part on the sensed information unit).

[0107] c) The field of view sensed by the sensor.

[0108] d) A field of view associated with a processing operation. The processing operation may include generating information based at least in part on the sensed information unit.

[0109] e) The rate at which frames are acquired by the sensor.

[0110] f) The frame rate associated with the processing operation.

[0111] g) Selection of one or more resources for performing perception-related processes.

[0112] h) The operation mode of resources used during the execution of perception-related processes, etc.

[0113] i) A region of interest within a sensing information unit and / or a region of interest within a processed sensing information unit.

[0114] As described above, a processing operation refers to any processing operation applied to a sensing information unit sensed by a sensor or any information generated at least in part based on the sensing information unit.

[0115] There may be discrepancies between sensor parameters and processing parameters. For example, a sensor may acquire 60 images per second while a processor processes fewer than 60 images. Alternatively, the processor processes more than 60 images (which may require generating additional images—for example, by estimating which images may have been acquired between consecutive images). As another example, the resolution of an image may change between image acquisition and processing.

[0116] A resource may be one or more narrow artificial intelligence agents, one or more processing circuits, a memory unit, etc.

[0117] According to one embodiment, the perception-related process does not include steps 210 and 220, and the determination of step 230 affects any response associated with the identified scenario (except for step 240).

[0118] According to one embodiment, the perception-related process affects the execution of future iterations of step 210 and / or step 220 .

[0119] According to one embodiment, step 230 includes at least one of steps 231 , 232 , 233 and 234 .

[0120] According to one embodiment, step 231 includes determining resource operating parameters via a machine learning process that is trained to infer resource operating parameters based on the identified context.

[0121] According to one embodiment, step 220 is performed by a first machine learning process and step 231 is performed by a second machine learning process different from the first machine learning process.

[0122] According to one embodiment, step 232 includes determining resource operating parameters based on applicable performance indicators that can be applied to the operation of the sensing-related process to achieve a specified performance. Applicable performance indicators may include, for example, desired power consumption and / or desired resolution.

[0123] According to one embodiment, step 233 comprises determining the resource operating parameter according to an applicable trade-off indicator indicating a trade-off between resource consumption and perception accuracy of the perception-related process.

[0124] According to one embodiment, step 230 includes step 234 of determining a resource operating parameter based on a rate of change within the environment. Method 200 may include receiving an estimate of the rate of change or making an estimate of the rate of change.

[0125] According to one embodiment, the rate of change is estimated based on the propagation speed of the vehicle and / or the environment of the vehicle. It can be expected that the rate of change in the environment when driving through a dense and highly populated urban environment exceeds the rate of change in the environment when driving through a deserted highway.

[0126] According to one embodiment, changes within the environment may include the inclusion of road users within the environment, the exclusion of road users from the environment, any changes in the behavior of road users within the environment, any changes in traffic conditions, any changes associated with the path of the vehicle (e.g., curves, junctions), etc. According to one embodiment, the rate of change is estimated based on changes that are expected to affect the progress of the vehicle (e.g., pedestrians entering the path of the vehicle, changes in the smoothness of the road, etc.).

[0127] According to one embodiment, the rate of change is estimated based on previous changes that occurred in the environment, previous changes that occurred in similar environments.Previous changes may have occurred in the presence of the vehicle and / or in the presence of other vehicles.

[0128] According to one embodiment, step 230 includes step 235 of determining resource operating parameters based on information related to road users within the scenario. According to one embodiment, the environmental information includes information related to road users. According to one embodiment, the environmental information is further processed to provide information related to road users.

[0129] According to one embodiment, step 235 is based on the detected influence of road users on the driving of the vehicle.

[0130] According to one embodiment, step 235 comprises (or is preceded by) detecting the road user by finding a match between characteristics of the road user and cluster characteristics of a cluster associated with the road user.

[0131] According to one embodiment, step 230 is followed by step 240 of responding to the determination of the resource operating parameter.

[0132] According to one embodiment, step 240 includes at least one of the following:

[0133] a) Step 241, making resource operating parameters available for operation of the perception-related process. According to one embodiment, step 241 includes making the resource operating parameters available by storing the resource operating parameters in a memory unit accessible by processing circuitry that controls at least some aspects of the perception-related process. Step 241 may trigger selection of one or more narrow AI agents from a set of narrow AI agents.

[0134] b) Step 242: Execute a perception-related process. According to one embodiment, the execution includes generating one or more narrow AI agent driving-related recommendations. According to one embodiment, the execution includes triggering and / or determining and / or recommending and / or instructing and / or executing a driving-related action based on the one or more narrow AI agent driving-related recommendations.

[0135] According to one embodiment, the resource operating parameters specify the selection of one or more narrow AI agents from a set of narrow AI agents.

[0136] According to one embodiment, step 240 triggers determining and / or suggesting and / or instructing and / or performing a driving-related action based on one or more narrow AI agent driving-related suggestions.

[0137] According to one embodiment, step 240 is followed by step 250 of triggering and / or determining and / or suggesting and / or instructing and / or performing a driving-related action based on the one or more narrow AI agent driving-related suggestions.

[0138] Figure 3 Examples of various perception modules, MDI generators, selection units, narrow AI agents, and driving decision units are shown.

[0139] The perception module includes:

[0140] i. Road setting perception module 361-1.

[0141] ii. Road user perception module 361-2.

[0142] iii. Traffic rules perception module 361-3.

[0143] iv. Environmental condition perception module 361-4.

[0144] v. Vehicle kinematic perception module 361-5

[0145] Figure 3 It may also include unselected perception modules that may be idle.

[0146] Selection unit 587 identifies one or more narrow AI agents by comparing MDI 585 to MDI cluster features 586 ( 1 )- 586 (J) to find one or more matching MDI clusters associated with the one or more narrow AI agents to be selected.

[0147] The matching MDI cluster indicates the narrow AI agent (denoted as 588) that should be selected. Figure 4 Also shown are the non-selected narrow AI agents (represented as including a dashed interior pattern).

[0148] The selected narrow AI agent 588 outputs a narrow AI agent driving decision 589 .

[0149] The driving decision unit 590 is configured to receive the narrow AI agent driving decision 589 and generate and output an output driving decision 591 that can be executed by one or more units of the vehicle.

[0150] Figure 4 An example of an image 610 acquired by a vehicle sensor is shown.

[0151] Image 610 shows two lanes 611 and 612 of a road, a right-side building 613, a first vehicle 615, a second vehicle 616, a tree 618, a table 617, a left-side building 621, a child 623, a ball 624, a traffic sign 630 indicating a maximum speed limit (e.g., 50 km / h), a driving direction arrow 634, a zebra crossing 633, and a crowd 628 in the process of reaching the zebra crossing and crossing the zebra crossing from right to left. Figure 4 Also shown are a progress (behavior) 641 of the child 623 , a progress 642 of the group 628 , a progress 645 of the first vehicle 615 , and a progress 646 of the second vehicle 616 .

[0152] The road user information may indicate movable road users, such as a first vehicle 615 , a second vehicle 616 , a child 623 , and a crowd 628 .

[0153] The road setting information may indicate static objects selected from lanes 611 and 612, road 610, tree 618, table 617, right building 613 and left building 621 - not all of these static objects are taken into account as not all of these static objects affect the vehicle.

[0154] The traffic regulation indication information indicates traffic signs 630, driving direction arrows 634, and lane boundaries. Traffic regulation information may also be learned from information not included in the image—for example, a data structure storing traffic regulations applicable to the vehicle's location.

[0155] The environmental condition information may indicate that the vehicle's environment is illuminated by solar radiation and is not obscured by rain or other weather elements.

[0156] The ego vehicle kinematic information may be learned using kinematic sensors (e.g., velocity sensors, acceleration sensors, etc.). Additionally or alternatively, this information may be learned by processing images acquired by the vehicle—performing visual odometry.

[0157] The rate of change in the environment is affected by the speed of the vehicles within the urban environment, the second vehicle 616 driving ahead of the ego vehicle, the first vehicle 615 driving in the opposite lane, thus preventing a bypass of the first vehicle, the lack of through lanes and intersections near the vehicles, and the presence of a child 623 on the left side of the road and a group of people crossing the road 638. The child 623 moving toward the road—following the ball—and causing the first vehicle 615 to advance toward the ball's location may indicate that the first vehicle may change its course—and even move ahead of the oncoming vehicle.

[0158] When acquiring the image, more resources may be allocated to the image segments surrounding the second vehicle 616, the first vehicle 615, and the child 623, and fewer resources should be allocated to other portions of the image. At a later point in time, more processing resources may be allocated to processing the crowd 638 crossing the road.

[0159] Figure 5 An image captured upon reaching a tunnel is shown, and resources should be allocated to a small segment 661 of the image including the lanes within the tunnel and the end of the tunnel. The tunnel may be dark, but for simplicity of explanation, the image segments outside the small segment 661 are white.

[0160] In the foregoing detailed description, numerous specific details have been set forth in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail in order to avoid obscuring the present invention.

[0161] The subject matter relating to the invention is particularly pointed out and distinctly claimed in the concluding portions of the specification. However, the invention as to its organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read in connection with the accompanying figures.

[0162] It should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. Furthermore, where deemed appropriate, reference numerals may be repeated in the drawings to indicate corresponding or similar elements.

[0163] Because the illustrated embodiments of the present invention can be implemented, for the most part, using electronic components and circuits known to those skilled in the art, details will not be explained to a greater extent than deemed necessary as shown above in order to understand and appreciate the basic concepts of the invention, and in order not to obscure or distract from the teachings of the invention.

[0164] Any reference to a method in this specification should be applied mutatis mutandis to an apparatus or system capable of performing the method and / or a non-transitory computer-readable medium storing instructions for performing the method.

[0165] Any reference in this specification to a system or apparatus should be applied mutatis mutandis to a method executable by the system and / or to a non-transitory computer readable medium storing instructions executable by the system.

[0166] Any reference in this specification to a non-transitory computer readable medium should be applied mutatis mutandis to an apparatus or system capable of executing the instructions stored in the non-transitory computer readable medium and / or should be applied mutatis mutandis to a method for executing the instructions.

[0167] Any combination of any modules or units listed in any figure, any part of the specification and / or any claims may be provided.

[0168] Any of the conversion module, active learning module, or clustering module, or any other module described herein, may be implemented in hardware and / or in code, instructions, and / or commands stored in a non-transitory computer-readable medium, and may be included in a vehicle, external to a vehicle, in a mobile device, in a server, or the like.

[0169] The vehicle may be any type of vehicle, such as a ground transportation vehicle, an air vehicle, or a water vehicle.

[0170] This specification and / or accompanying drawings may refer to a processor. A processor may be a processing circuit. The processing circuit may be implemented as a central processing unit (CPU), and / or one or more other integrated circuits, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a full-custom integrated circuit, etc., or a combination of such integrated circuits.

[0171] Any combination of any steps of any method shown in the description and / or drawings may be provided.

[0172] Any combination of any subject matter of any claim may be provided.

[0173] Any combination of the systems, units, components, processors, sensors shown in the specification and / or drawings may be provided.

[0174] Further examples are listed below.

[0175] Embodiment 1. A computer-implemented method for a perception-related process, the method comprising: receiving, by processing circuitry of a vehicle, scene information about a scene facing the vehicle, wherein the scene information includes environmental information about an environment of the vehicle; identifying, by the processing circuitry, the scene using the received scene information; determining, based on the identified scene, resource operating parameters that are consistent with the identified scene and relevant to operation of the perception-related process; and making the resource operating parameters available to the operation of the perception-related process.

[0176] Embodiment 2. The method of embodiment 1, wherein the determination of the resource operating parameter is based on applicable performance indicators applicable to the operation of the sensing-related process to achieve a specified performance.

[0177] Embodiment 3. The method according to embodiments 1 and 2, wherein the determination of the resource operation parameter is in accordance with an applicable trade-off indicator indicating a trade-off between resource consumption of the perception-related process and perception accuracy.

[0178] Embodiment 4. The method of embodiments 1-3, wherein the resource operation parameter specifies selection of a narrow AI perception module from a set of narrow AI perception modules.

[0179] Embodiment 5. The method of embodiments 1-4, further comprising estimating a rate of change within the environment, wherein the determination of the resource operating parameter is further based on the rate of change within the environment.

[0180] Embodiment 6. The method of embodiments 1-5, wherein the estimate of the rate of change is responsive to a propagation speed of the vehicle.

[0181] Embodiment 7. The method of embodiments 1-6, wherein the estimate of the rate of change is further responsive to the environment.

[0182] Embodiment 8. The method of embodiments 1-7, further comprising obtaining information related to road users within the scene, wherein the determination of the resource operating parameters is further responsive to the detected road users.

[0183] Embodiment 9. The method of embodiments 1-8, wherein the determination of the sensing solution is further responsive to a detected impact of a road user on the driving of the vehicle.

[0184] Embodiment 10. The method of embodiments 1-9, wherein the detection of the road user comprises finding a match between a signature of the road user and a cluster signature of a cluster associated with the road user.

[0185] Embodiment 11. The method of embodiments 1-10, wherein identifying the scene comprises finding a match between features of a portion of the scene information and cluster features of a cluster associated with the scene.

[0186] Embodiment 12. A non-transitory computer-readable medium for a perception-related process, the non-transitory computer-readable medium storing instructions that, when executed by a computerized system, cause the computerized system to: receive, by processing circuitry of a vehicle, scene information about a scene facing the vehicle, wherein the scene information includes environmental information about an environment of the vehicle; identify, by the processing circuitry, the scene information using the received scene information; determine, based on the identified scene, resource operating parameters that are consistent with the identified scene and relevant to operation of the perception-related process; and make the resource operating parameters available to the operation of the perception-related process.

[0187] Embodiment 13. The non-transitory computer readable medium of embodiment 12, wherein the determination of the resource operating parameter is based on applicable performance indicators applicable to the operation of the sensing-related process to achieve a specified performance.

[0188] Embodiment 14. The non-transitory computer readable medium of embodiments 12 and 13, wherein the determination of the resource operating parameter is in accordance with an applicable trade-off indicator indicating a trade-off between resource consumption of the perception-related process and perception accuracy.

[0189] Embodiment 15. The non-transitory computer-readable medium of embodiments 12-14, wherein the resource operation parameter specifies selection of a narrow AI perception module from a set of narrow AI perception modules.

[0190] Embodiment 16. The non-transitory computer readable medium of embodiments 12-15, further storing instructions for estimating a rate of change within an environment, wherein the determination of the resource operating parameter is further based on the rate of change within the environment.

[0191] Embodiment 17. The non-transitory computer readable medium of embodiments 12-16, wherein the estimate of the rate of change is responsive to at least one of a propagation speed of the vehicle or the environment.

[0192] Embodiment 18. The non-transitory computer-readable medium of embodiments 12-17, further storing instructions for obtaining information related to road users within the scene, wherein the determination of the resource operating parameter is further responsive to the detected road users.

[0193] Embodiment 19. A computerized system for a perception-related process, the computerized system comprising: a memory unit configured to store scenario information about a scenario facing a vehicle, wherein the scenario information includes environmental information about the vehicle's environment; and processing circuitry configured to: identify a scenario using the received scenario information; determine, based on the identified scenario, resource operating parameters that are consistent with the identified scenario and relevant to operation of the perception-related process; and make the resource operating parameters available to operation of the perception-related process.

Claims

1. A computer-implemented method for sensing a related process, the method comprising: Receiving, by a processing circuit of a vehicle, scene information of a scene faced by the vehicle, wherein the scene information includes environmental information of an environment of the vehicle; identifying, by the processing circuit, the scene using the received scene information; determining, based on the identified scenario, resource operating parameters consistent with the identified scenario and associated with operation of the perception-related process; as well as The resource operation parameters are made available to the operation of the perception-related process.

2. The method of claim 1 , wherein the determining of the resource operation parameter is based on one or both of an applicable performance indicator that can be applied to the operation of the perception-related process to achieve a specified performance, and an applicable trade-off indicator that indicates a trade-off between resource consumption and perception accuracy of the perception-related process.

3. The method of claim 1, wherein the resource operating parameter specifies selection of a narrow AI perception module from a set of narrow AI perception modules. 4 . The method of claim 1 , further comprising estimating a rate of change within the environment, wherein the determining of the resource operating parameter is further based on the rate of change within the environment. 5 . The method of claim 1 , further comprising obtaining information related to road users within the scenario, wherein the determining of the resource operating parameters is also responsive to the detected road users.

6. The method of claim 5, wherein the determination of the sensing solution is further responsive to detected influence of road users on driving of the vehicle. 7 . The method of claim 5 , wherein the detecting of the road user comprises finding a match between characteristics of the road user and cluster characteristics of a cluster associated with the road user.

8. The method of claim 5, wherein said identifying the scene comprises finding a match between features of a portion of the scene information and cluster features of a cluster associated with the scene.

9. A non-transitory computer-readable medium for use in a perception-related process, the non-transitory computer-readable medium storing instructions that, upon execution by a computerized system, cause the subject computerized system to: Receiving, by a processing circuit of a vehicle, scene information of a scene faced by the vehicle, wherein the scene information includes environmental information of an environment of the vehicle; identifying, by the processing circuit, the scene using the received scene information; determining, based on the identified scenario, resource operating parameters consistent with the identified scenario and associated with operation of the perception-related process; as well as The resource operation parameters are made available to the operation of the perception-related process.

10. A computerized system for use in a perception-related process, the computerized system comprising: a memory unit configured to store scene information about a scene faced by the vehicle; wherein the scene information includes environmental information about the environment of the vehicle; and The processing circuit is configured to: identifying the scene using the received scene information; determining, based on the identified scenario, resource operating parameters consistent with the identified scenario and associated with operation of the perception-related process; as well as The resource operation parameters are made available to the operation of the perception-related process.

Images