MACHINE WITH A PERCEPTION SENSOR SYSTEM CONTAINING MULTIPLE SETS OF ONE OR MORE SENSORS

Abstract

A controller (114) of a machine (100) can identify that an attachment (110) and a linkage (122) of the machine (100) are in a first position. Based on this identification, the controller (114) can determine that the attachment (110) and the linkage (122) are in the first position and do not block a first sensing zone (206) associated with a first set of one or more sensors (202) of a sensing sensor system (112) of the machine (100). Based on the determination that the implement (110) and the linkage (122) do not block the first sensing zone (206), the control unit (114) can cause initial sensing data acquired by the first set of one or more sensors (202) to be used in conjunction with an automatic control operation.

Description

Technical field

[0001] The present disclosure relates in general to a machine and, for example, a machine with a perception sensor system comprising several sets of one or more sensors. State of the art

[0002] To perform an unloading operation, a machine, such as a wheel loader, can use an attachment (e.g., a bucket or other implement) to load and transport material (e.g., asphalt, deposits, soil, snow, feed, gravel, logs, raw minerals, recycled material, rock, sand, wood chips, or similar material) and unload the material into a discharge destination (e.g., another machine, such as a dump truck). The machine may incorporate one or more sensors to detect the distance between the machine and the discharge destination, enabling an automatic attachment lifting operation.For example, as part of the automatic lifting process, the machine can use one or more sensors to identify when the machine is close enough to the unloading target and thereby automatically cause the attachment to lift to a unloading height to enable efficient unloading of material from the attachment into the unloading target (e.g., when the machine reaches the unloading target).

[0003] In many cases, however, the geometry of the attachment and the linkage connecting it to the machine can obstruct the field of view of one or more sensors, preventing the machine from accurately determining the distance to the unloading point. This can often lead to the machine unintentionally coming into contact with the unloading point, for example, because the stopping distance or the position of the attachment relative to the unloading point was miscalculated. This, in turn, causes damage (e.g., dents, cracks, or other types of structural deformation) to the attachment and linkage, the machine, and the unloading point. This can negatively impact the performance and lifespan of the attachment, linkage, machine, and unloading point.

[0004] The control system described in this disclosure solves one or more of the problems listed above and / or other prior art problems. Brief description

[0005] A machine comprises: an attachment and linkage associated with the front of the machine; a sensing sensor system comprising a first set of one or more sensors and a second set of one or more sensors associated with the front of the machine; and a controller designed to: identify that the attachment and linkage are in a first position; determine, based on identifying that the attachment and linkage are in the first position, that the attachment and linkage do not block a first sensing zone associated with the first set of one or more sensors;and induce, based on the determination that the implement and linkage do not block the first sensing zone, that first sensing data acquired by the first set of one or more sensors is used in conjunction with an automatic control operation assigned to at least the machine or implement and linkage.

[0006] A machine controller includes one or more memories; and one or more processors coupled to the one or more memories and designed to: identify a field of view of a first set of one or more sensors of a machine's perception sensor system; cause an attachment and linkage of the machine to move over a range of positions; determine, for each position in the range of positions, whether the attachment and linkage obstruct the field of view; identify a set of one or more positions in the range of positions associated with the attachment and linkage that do not obstruct the field of view; and identify a first perception zone of the perception sensor system associated with the set of one or more positions.A procedure involves identifying, by a machine's control system, that an attachment and linkage of the machine are in a first position; determining, by the control system and based on the identification that the attachment and linkage are in the first position, that the attachment and linkage do not block a first sensing zone associated with a first set of one or more sensors of a sensing sensor system of the machine; and causing, by the control system and based on the determination that the attachment and linkage do not block the first sensing zone, that first sensing data acquired by the first set of one or more sensors are used in conjunction with an automatic control operation. Brief description of the drawings Fig. Figure 1 is a representation of an exemplary machine described herein. Fig. Figure 2 shows an example configuration of the front of the machine. Fig. Figures 3A-3C are representations of an exemplary implementation described herein. Fig. Figures 4A-4B are representations of an exemplary implementation described herein. Fig. Figures 5A-5B are representations of an exemplary implementation described herein. Fig. Figure 6 is a representation of exemplary components of a device that are assigned to a machine's perception sensor system. Detailed description

[0007] This disclosure relates to the control of a machine (which, for example, performs an unloading operation) and is applicable to any machine capable of loading and moving material (e.g., from one location to another) and / or unloading the material (e.g., into a delivery destination). The machine could be, for example, any machine that performs an operation related to an industry such as mining, construction, agriculture, transportation, or any other industry.In some examples, the machine may be a vehicle, wheel loader, backhoe loader, cold cutter, compactor, felling grapple, forestry machine, loader, harvester, excavator, industrial loader, articulated loader, material handler, motor grader, pipelayer, road conditioner, skid steer loader, tractor, bulldozer, tractor scraper, or other aboveground equipment, underground equipment, aerial equipment, or marine equipment.

[0008] Fig. Figure 1 is a representation of an exemplary machine 100 described herein. For example, machine 100 could be a mobile machine, such as the one described in Fig. The machine shown may include the wheel loader shown, or any other type of mobile machine. Furthermore, the machine may be manned or unmanned and / or fully autonomous, semi-autonomous, or remotely controlled.

[0009] As shown, the machine 100 can have a frame 102 that supports an operator station 104, a power system 106, a drive system 108, an attachment 110, a perception sensor system 112, and a control unit 114. The operator station 104 can include operator controls 116 for operating the machine 100 via the power system 106. In some examples, the machine 100 may not include an operator station 104 and / or operator controls 116 (e.g., the machine 100 may be controlled by other means, such as a remote control system). The operator station 104 can be designed to define an interior cabin 118 in which the operator controls 116 are housed.

[0010] The power system 106 is designed to supply power to the machine 100. The power system 106 can be configured to receive control signals from the operator controls 116 in the operator station 104 and / or from the controller 114. Additionally or alternatively, the power system 106 can be configured to operate the drive system 108 and / or the attachment 110 selectively according to the control signals. The power system 106 can provide operating power for driving the drive system 108 and / or operating the attachment 110. The power system 106 can include a drive, a motor, an electric drive, a fuel cell, and / or another type of power system.

[0011] The drive system 108 can be operationally arranged with the power system 106 to selectively drive the machine 100 via the control signals. The drive system 108 can include a variety of ground engagement elements, such as wheels 120, as shown, which can be movably connected to the frame 102 via axles, drive shafts, and / or other components. The drive system 108 can be in the form of a chain drive system, a wheel drive system, or another type of drive system designed to drive the machine 100.

[0012] The attachment 110 can be arranged in an operational configuration with the power system 106, so that the attachment 110 can be selectively moved by control signals transmitted from the operator controls 116 and / or the control unit 114 to the power system 106. As shown in Fig. As shown in Figure 1, the attachment 110 can be coupled to the machine 100 via a linkage 122, for example at the front 124 of the machine 100. The attachment 110 can also be referred to as an attachment, working tool, implement, and / or tool. Fig. Figure 1 shows the attachment 110 as an example in the form of a shovel. Other embodiments may include any attachment 110 suitable for a variety of tasks, such as, but not limited to, grading, sweeping, compacting, leveling, lifting, loading, plowing, and / or ripping. Exemplary attachments 110 include, but are not limited to, a stump grinder, a trenching machine, a sweeper, a brush cutter, a cold planer, a mower, a mulcher, a processor, a pulverizer, a rake, a saw, a snow gun, a snowblower, a cultivator, a winch, an auger, a shovel, a crusher / hammer, a compactor, a cutter, a forklift, a grader and end chisel, a grapple, a bucket, and / or a ripper. As described elsewhere herein, the attachment 110 may include one or more components or parts that are electrically powered.

[0013] The perception sensor system 112 includes a variety of sensors 126 that can be coupled to the machine 100, for example, on the front 124 of the machine 100. The variety of sensors 126 can include a sonar sensor, an acoustic sensor, a camera, a light detection and distance measurement sensor (LIDAR sensor), and / or a radio detection and distance measurement sensor (RADAR sensor), or another type of sensor to perceive the environment of the machine 100. That is, the variety of sensors 126 can include at least one sensor designed to acquire perception data that can be used (e.g., by the controller 114) to determine at least one of the distance to a target (e.g., an unloading target) or the height of the target. The variety of sensors 126 can also include a position sensor (e.g.,a global positioning system (GPS) sensor or a local positioning system sensor) designed to determine a physical location of the machine 100, a position sensor (e.g. a rotation sensor or other sensor) designed to detect a position of the attachment 110 and the linkage 122, a speed sensor designed to determine a speed of the machine 100 (e.g. when driving over a surface), and / or include one or more other sensors.

[0014] The controller 114 may include an electronic control module (ECM) or other computer device. The controller 114 may be configured to cause perception data acquired by sets of one or more sensors of the perception sensor system 112 to be used in conjunction with an automatic control operation associated with at least one of the machine 100 or the attachment 110 and the linkage 122, to cause an automatic perception zone calibration operation to be performed (e.g., in conjunction with the perception sensor system 112), and / or to cause one or more other operations to be performed, as further described herein.

[0015] A rear section 128 of the machine 100 can include a drive and a transmission. The drive can be any type of drive suitable for performing work with the machine 100, such as an internal combustion engine, a diesel engine, a gasoline engine, a gaseous fuel-powered engine, and / or the like. In other examples, the machine 100 can include a different power system instead of a drive, such as a motor (e.g., an electric motor), a battery-powered system, a fuel cell, or another type of power system. The transmission can transfer power from the drive to the drive system 108 and / or the attachment 110.

[0016] As mentioned above, Fig. 1 is provided as an example. Other examples may differ from what is associated with Fig. 1 is described. Fig. Figure 2 is a representation of an exemplary configuration 200 of the front 124 of the machine 100 (e.g., when the attachment 110 and the linkage 122 are extended to a “high” position, as further described herein). The perception sensor system 112 can include a variety of sensors 126, which are subdivided into several sets of one or more sensors (e.g., those assigned to the front 124 of the machine 100). For example, the perception sensor system 112, as shown in Fig. Figure 2 shows a plurality of sensors 126, which are assigned to a first set of one or more sensors 202 and a second set of one or more sensors 204. The multiple sets of one or more sensors can be positioned at different heights (e.g., on the machine 100, on the front 124 of the machine 100). For example, as shown in Fig. As shown in Figure 2, the first set of one or more sensors 202 may be positioned at a first height that is associated with a top surface of the operator station 104 (e.g. aligned with it or nearly aligned with it), and the second set of one or more sensors 204 may be positioned at a second height that is associated with the wheels 120 (e.g. aligned with them or nearly aligned with them).

[0017] A set of one or more sensors can include sensors of different types. For example, the first set of one or more sensors 202 or the second set of one or more sensors 204 can include a first sensor of a first type and a second sensor of a second type, the first type being different from the second type. As a specific example, the first set of one or more sensors 202 can include a radar sensor and a camera. By including a radar sensor, the first set of one or more sensors 202 can be able to accurately detect a target (e.g., a drop-off target) at long distances (e.g., distances of more than 6 meters), regardless of the lighting conditions to which the machine 100 is exposed (e.g.,(whether the machine 100 operates in full sun, partial sun, no sun, artificial lighting, and / or without artificial lighting), and by including the camera, the first set of one or more sensors 202 can be able to accurately identify a marker (e.g., a reference point) that distinguishes the target from another object. As another specific example, the second set of one or more sensors 204 can include a radar sensor and a sonar sensor. By including a radar sensor, the second set of one or more sensors 204 can be able to accurately detect a target (e.g., an unloading target) at long distances (e.g., distances of more than 6 meters), and by including the sonar sensor, the second set of one or more sensors 204 can be able to accurately detect a target at short distances (e.g., distances of less than or equal to 6 meters).

[0018] Additionally or alternatively, a set of one or more sensors can include multiple sensors of the same type. For example, the first set of one or more sensors 202 or the second set of one or more sensors 204 can include a first sensor of a first type and a second sensor of the first type. As a specific example, the first set of one or more sensors 202 can include multiple RADAR sensors (e.g., positioned on both sides of the operator station 104). By including multiple RADAR sensors, the field of view of the first set of one or more sensors 202 can be increased (e.g., extended horizontally), and the detection accuracy and precision of the first set of one or more sensors 202 can be improved. As another specific example, the second set of one or more sensors 204 can include multiple sonar sensors arranged in a two-dimensional layout (e.g.,The sonar sensors are arranged in a row and column layout or in a one-dimensional layout (e.g., column layout or row layout). By including multiple sonar sensors, the field of view of the second set of one or more sensors 204 can be increased (e.g., widened horizontally and / or extended vertically), and the detection accuracy and precision of the second set of one or more sensors 204 can be improved.

[0019] Furthermore, the perception sensor system 112 can include a variety of perception zones. A perception zone can be assigned to one or more sets of sensors. A perception zone can be a portion of a field of view (e.g., an aggregated field of view) of one or more sensors, within which the one or more sensors are able to accurately perceive the environment of the machine 100 (e.g., for specific positions of the attachment 110 and the linkage 122). The perception zone can have a three-dimensional shape, such as a cone, a sphere, a rectangular prism, or another type of shape.As a specific example, the perception sensor system 112 can have a first perception zone 206, which is associated with the first set of one or more sensors 202, and a second perception zone 208, which is associated with the second set of one or more sensors 204. At least some of the multiple perception zones may overlap, or alternatively, the multiple perception zones may not overlap. Since the multiple sets of one or more sensors may include sensors of different types, each perception zone can be a composite of incongruent but similarly arranged individual sensor zones.

[0020] As stated above, Fig. 2 is provided as an example. Other examples may differ from what is associated with Fig. 2 is described.

[0021] Fig. Figures 3A-3C are representations of an exemplary implementation 300 described herein. Fig. Figures 3A-3C show side views of the machine 100 when the attachment 110 and the linkage 122 are in different positions.

[0022] As in Fig. As shown in Figure 3A, the attachment 110 and the linkage 122 can be in a "low" position (e.g., when the attachment 110 is aligned with or nearly aligned with the wheels 120). As part of an automatic control operation (which is assigned, for example, to at least one of the machines 100 or to the attachment 110 and the linkage 122), the controller 114 can cause the attachment 110 and the linkage 122 to be in the lower position. For example, as part of the automatic control operation, the controller 114 can cause the attachment 110 and the linkage 122 to be in the lower position to allow loading and / or transporting material, such as from a first location to a second location assigned to an unloading destination (e.g., another machine, such as a dump truck). As shown in Figure 3A, the attachment 110 and the linkage 122 can be in a "low" position (e.g., when the attachment 110 is aligned with or nearly aligned with the wheels 120). Fig. As shown in Figure 3A, the implement 110 and the linkage 122, in the lowered position, cannot block the first perception zone 206 of the perception sensor system 112, which is assigned to the first set of one or more sensors 202, but can block the second perception zone 208 of the perception sensor system 112, which is assigned to the second set of one or more sensors 204. Accordingly, the controller 114 can cause the first perception data, acquired by the first set of one or more sensors 202, to be used in conjunction with the automatic control operation (e.g., it can allow the first perception data to be used as input for the automatic control operation), and / or it can cause the second perception data, acquired by the second set of one or more sensors 204, not to be used in conjunction with the automatic control operation (e.g., it can allow the first perception data to be used as input for the automatic control operation).It can prevent the second perception data from being used as input for the automatic control process), as further described herein.

[0023] As in Fig. As shown in Figure 3B, the attachment 110 and the boom 122 can be in a “middle” position (e.g., when the attachment 110 is aligned with or nearly aligned with the underside of the operator station 104). As part of the automatic control process, the controller 114 can cause the attachment 110 and the boom 122 to be in the middle position. For example, as part of the automatic control process, the controller 114 can cause the attachment 110 and the boom 122 to be in the middle position when the machine 100 is sufficiently close (e.g., within a first threshold distance) to the unloading destination (e.g., to facilitate the possible unloading of material carried by the attachment 110). As shown in Figure 3B, the attachment 110 and the boom 122 can be in the middle position when the machine 100 is sufficiently close (e.g., within a first threshold distance) to the unloading destination (e.g., to facilitate the possible unloading of material carried by the attachment 110). Fig. As shown in Figure 3B, the implement 110 and the linkage 122, in the middle position, can block the first sensing zone 206, but not the second sensing zone 208. Accordingly, the controller 114 can cause the first sensing data, acquired by the first set of one or more sensors 202, not to be used in conjunction with the automatic control operation (e.g., it can prevent the first sensing data from being used as input for the automatic control operation) and / or it can cause the second sensing data, acquired by the second set of one or more sensors 204, to be used in conjunction with the automatic control operation (e.g., it can allow the second sensing data to be used as input for the automatic control operation), as further described herein.

[0024] As in Fig. As shown in Figure 3C, the attachment 110 and the boom 122 can be in a “high” position (e.g., when the attachment 110 is aligned with or nearly aligned with the top of the operator station 104). As part of the automatic control process, the controller 114 can cause the attachment 110 and the boom 122 to be in the high position. For example, as part of the automatic control process, the controller 114 can cause the attachment 110 and the boom 122 to be in the high position when the machine 100 is within a “discharge distance” (e.g., within a second threshold distance that is less than the first threshold distance) of the discharge destination (e.g., to facilitate the imminent discharge of the material transported by the attachment 110). As shown in Figure 3C, the attachment 110 and the boom 122 can be in the high position when the machine 100 is within a “discharge distance” (e.g., within a second threshold distance that is less than the first threshold distance) of the discharge destination (e.g., to facilitate the imminent discharge of the material transported by the attachment 110). Fig. As shown in Figure 3C, the implement 110 and the linkage 122, in the high position, can block the first sensing zone 206, but not the second sensing zone 208. Accordingly, the controller 114 can cause the first sensing data, acquired by the first set of one or more sensors 202, not to be used in conjunction with the automatic control operation (e.g., it can prevent the first sensing data from being used as input for the automatic control operation) and / or it can cause the second sensing data, acquired by the second set of one or more sensors 204, to be used in conjunction with the automatic control operation (e.g., it can allow the second sensing data to be used as input for the automatic control operation), as further described herein.

[0025] While Fig. Figures 3A-3C show specific examples of sets of sensors (e.g., positioned at specific heights and locations on the machine 100), of sensing zones (e.g., with specific geometries that do not overlap), and of the attachment 110 and the linkage 122 that block or do not block the sensing zones, based on specific positions of the attachment 110 and the linkage 122. Implementations include any combination of sets of sensors (e.g., positioned at any height and location on the machine 100), of sensing zones (e.g., with any geometry, whether overlapping or not overlapping with other sensing zones), or of the attachment 110 and the linkage 122 that block or do not block the sensing zones, based on any position of the attachment 110 and the linkage 122.Accordingly, the controller 114 can, based on any configuration of a set of one or more sensors, a perception zone associated with the set of one or more sensors, and a position of the implements 110 and the linkage 122, cause (e.g., selectively cause) perception data acquired by the set of one or more sensors to be used or not used in conjunction with the automatic control process, as further described herein.

[0026] In some implementations, the controller 114 can cause the attachment 110 and the linkage 122 to move over a range of positions, for example in conjunction with an automatic sensing zone calibration process, as described herein in relation to the Fig. 5A-5B further described. For example, the control 114 can cause the attachment 110 and the linkage 122 to move into the lower position, the middle position, the upper position and / or into one or more other positions (e.g. when the attachment 110 and the linkage 122 are pivoted).

[0027] As stated above, the Fig. 3A to 3C are provided as examples. Other examples may differ from what is shown in connection with Fig. 3A to 3C is described. Fig. Figures 4A-4B are representations of an exemplary implementation 400 described herein. Fig. Figures 4A-4B show how the controller 114 facilitates the selection of perception data acquired by the multitude of sensors 126 of the perception sensor system 112 for use in conjunction with an automatic control operation (e.g., one assigned to at least one of the machine 100 or the attachment 110 and / or the boom 122). The automatic control operation can, for example, cause the machine 100 to move to or from a target (e.g., a discharge point) and / or cause the attachments 110 and the boom to move into specific positions (e.g., based on the proximity of the machine 100 to the target).

[0028] As in Fig. As represented by reference number 4A and 402, the controller 114 can identify that the attachment 110 and the linkage 122 are in a first position. For example, if the plurality of sensors 126 includes a position sensor, the controller 114 can receive position data from the position sensor (e.g., in real time or near real time) and process the position data (e.g., analyze and / or read it, among other things) to determine that the attachment 110 and the linkage 122 are in the first position. The attachment 110 and the linkage 122 may, for example, be in the first position to load and / or transport material, such as from a first location to a second location associated with an unloading destination (e.g., another machine, such as a dump truck). The first position may, for example, be the lower position, the middle position, or the upper position, as defined herein with reference to the Fig. 3A-3C are described, or it could be another position.

[0029] As described in reference 404, the controller 114 can determine (e.g., based on identifying that the implement 110 and the linkage 122 are in the first position) whether the implement 110 and the linkage 122 are blocking a sensing zone associated with a set of one or more sensors of the plurality of sensors 126 of the sensing sensor system 112. To determine whether the implement 110 and the linkage 122 are blocking a sensing zone, the controller 114 can determine whether the first position is within a permissible set of one or more positions associated with the sensing zone (e.g., whether the first position is an element of the permissible set of one or more positions). The controller 114 can determine the permissible set of one or more positions in conjunction with an automatic sensing zone calibration process, as described herein with reference to the Fig. Described in sections 5A-5B. If the controller 114 determines that the first position is within the permissible set of one or more positions, the controller 114 can determine that the attachment 110 and the linkage 122 do not block the sensing zone (e.g., that the attachment 110 and the linkage 122 do not obscure the set of one or more sensors assigned to the sensing zone). If the controller 114 determines that the first position is not within the permissible set of one or more positions, the controller 114 can determine that the attachment 110 and the linkage 122 block the sensing zone (e.g., that the attachment 110 and the linkage 122 obscure a set of one or more sensors assigned to the sensing zone).

[0030] For example, with reference to Fig. 3. The controller 114 can determine, when the first position is the lower position, that the implement 110 and the linkage 122 do not block the first sensing zone 206, which is assigned to the first set of one or more sensors 202, and that the implement 110 and the linkage 122 do block the second sensing zone 208, which is assigned to the second set of one or more sensors 204. As a further example, with reference to Fig. 3B-3C, the controller 114 can determine that the attachment 110 and the linkage 122 block the first sensing zone 206 and that the attachment 110 and the linkage 122 do not block the second sensing zone 208 when the first position is the middle position or the high position.

[0031] As in Fig. As represented by reference number 4B and reference number 408, the controller 114 can cause (e.g., selectively cause) perception data acquired by a set of one or more sensors to be used or not used in conjunction with an automatic control operation (e.g., one associated with at least one of the machine 100 or the attachment 110 and the linkage 122), such as based on determining whether the attachment 110 and the linkage 122 block a perception zone associated with the set of one or more sensors. For example, based on determining that the attachment 110 and the linkage 122 do not block a perception zone associated with a set of one or more sensors, the controller 114 can cause perception data acquired by the set of one or more sensors to be used in conjunction with the automatic control operation (e.g.,to allow the perception data to be used as input for the automatic control operation). As an alternative example, the controller 114, based on determining that the implement 110 and the linkage 122 block a perception zone associated with a set of one or more sensors, can cause perception data acquired by the set of one or more sensors not to be used in conjunction with the automatic control operation (e.g., to prevent the perception data from being used as input for the automatic control operation).

[0032] As a specific example, with reference to Fig. 3A can cause the controller 114, based on the determination that the implement 110 and the linkage 122 do not block the first perception zone 206, which is assigned to the first set of one or more sensors 202, to use first perception data acquired by the first set of one or more sensors 202 in conjunction with the automatic control operation, and, based on the determination that the implement 110 and the linkage 122 block the second perception zone 208, which is assigned to the second set of one or more sensors 204, to not use second perception data acquired by the second set of one or more sensors 204 in conjunction with the automatic control operation.In some implementations, the controller 114 can determine that the implement 110 and the linkage 122 do not block sensing zones assigned to multiple sets of one or more sensors of the sensing sensor system 112. Accordingly, the controller 114 can cause the respective sensing data acquired by the multiple sensor sets to be used in conjunction with the automatic control operation. For example, if the first position is different from the lower, middle, and upper positions (e.g., as in the... Fig. (as shown in Figures 3A-3C), the controller 114 can determine that the implement 110 and the linkage 122 do not block the first sensing zone 206 and do not block the second sensing zone 208. The controller 114 can then cause the first sensing data, acquired by the first set of one or more sensors 202, and the second sensing data, acquired by the second set of one or more sensors 204, to be used in conjunction with the automatic control operation.

[0033] The control unit 114 can have one or more of these in relation to the Fig. The operations described in 4A-4B can be repeated, for example, at a later time (e.g., after causing certain perception data to be used in conjunction with the automatic control operation). For example, the controller 114 can determine that the implement 110 and the linkage 122 are in a second position (which differs, for example, from the first position), for example, in a manner similar to that described herein with reference to reference 402. The implement 110 and the linkage 122 can be in the second position as a result of causing the use of the certain perception data in conjunction with the automatic control operation (e.g., the automatic control operation can, by using the certain perception data, cause the implement 110 and the linkage 122 to move into the second position).

[0034] The controller 114 can then determine (e.g., based on identifying that the implement 110 and the linkage 122 are in the second position) whether the implement 110 and the linkage 122 are blocking a sensing zone associated with a set of one or more sensors of the plurality of sensors 126 of the sensing sensor system 112, similarly to how described herein with reference to reference 404. Accordingly, the controller 114 can cause (e.g., selectively cause) sensing data acquired by the set of one or more sensors to be used or not used in conjunction with the automatic control operation, similarly to how described herein with reference to reference 408.

[0035] For example, with reference to the Fig. 3A-3B, the attachment 110 and the linkage 122 can be moved from the lower position (as shown in Fig. 3A) as the first position. the middle position (as shown in Fig. (3B shown) as the second position. Accordingly, the controller 114 can, at a later time, when the attachment 110 and the linkage 122 are in the second position, determine that the attachment 110 and the linkage 122 block the first perception zone 206 (e.g., that the attachment 110 and the linkage 122 obscure the first perception zone 206) and that the attachment 110 and the linkage 122 do not block the second perception zone 208 (e.g., that the attachment 110 and the linkage 122 do not obscure the second perception zone 208).Accordingly, based on the determination that the implement 110 and the linkage 122 block the first perception zone 206, the controller 114 can cause first perception data acquired by the first set of one or more sensors 202 not to be used in conjunction with the automatic control operation, and based on the determination that the implement 110 and the linkage 122 do not block the second perception zone 208, it can cause second perception data acquired by the second set of one or more sensors 204 to be used in conjunction with the automatic control operation.

[0036] As stated above, the Fig. 4A to 4B are provided as examples. Other examples may differ from what is shown in connection with Fig. 4A to 4B are described. Fig. Figures 5A-5B are representations of an exemplary implementation 500 described herein. Fig. Figures 5A-5B show how the controller initiates an automatic perception zone calibration process to identify, for example, a perception zone of the perception sensor system 112 that is assigned to a set of one or more sensors. The perception zone can be assigned to a permissible set of one or more positions of the implement 110 and the boom 122, which includes positions of the implement 110 and the boom 122 that do not obstruct, or only minimally obstruct, the field of view of the set of one or more sensors. The automatic perception zone calibration can be performed prior to the automatic control process.

[0037] As in Fig. As represented by reference number 5A and reference number 502, the controller 114 can identify a field of view of a set of one or more sensors of the perception sensor system 112. For example, the controller 114 can identify a first field of view of the first set of one or more sensors 202 and / or a second field of view of the second set of one or more sensors 204. The controller 114 can identify a field of view of a set of one or more sensors based on configuration information associated with the set of one or more sensors (e.g., stored in a data structure accessible to the controller 114).

[0038] As in Fig. As shown in Figure 5B and reference number 504, the controller 114 can cause the attachment 110 and the linkage 122 to move through a range of positions. For example, the controller 114 can cause the attachment 110 and the linkage 122 to move to the lower position, the middle position, the upper position, and / or to one or more other positions (e.g., when pivoting the attachment 110 and the linkage 122). The controller 114 can cause the attachment 110 and the linkage 122 to move through the range of positions by sending one or more control signals to the power system 106.

[0039] As illustrated by reference 506, the controller 114 can determine, for each position within the range of positions, whether the implement 110 and the linkage 122 obstruct the field of view of the set of one or more sensors. For example, the controller 114 can cause the set of one or more sensors to acquire perception data (e.g., when the implement 110 and the linkage 122 are in each position), and the controller 114 can process the perception data (e.g., using one or more analysis techniques) to determine that the field of view is obstructed by the implement 110 and the linkage 122 (e.g., that at least a minimum percentage of the field of view is obstructed), or alternatively, that the field of view is not obstructed by the implement 110 and the linkage 122.

[0040] As illustrated by reference number 508, the controller 114 can identify a permissible set of one or more positions of the implement 110 and the linkage 122 that are associated with the set of one or more sensors (e.g., based on determining, for each position in the range of positions, whether the implement 110 and the linkage 122 obstruct the field of view of the set of one or more sensors). For example, the controller 114 can identify the permissible set of one or more positions as a set of one or more positions in the range of positions that are associated with the implement 110 and the linkage 122 and do not obstruct the field of view. That is, for a given position, the controller 114 can determine that the field of view is not obstructed by the implement 110 and the linkage (e.g.,It can determine that the entire field of view is not obstructed or that less than the minimum percentage of the field of view is obstructed. Accordingly, the control 114 can identify the specific position as one element of the permissible set of one or more positions.

[0041] As illustrated by reference 510, the controller 114 can identify a sensing zone of the sensing sensor system 112, which is associated with the set of one or more sensors, as being associated with the permissible set of one or more positions of the implements 110 and the linkage 122. Accordingly, if the implement 110 and the linkage 122 are in a position that is an element of the permissible set of one or more positions, the controller 114 can determine that the implement 110 and the linkage 122 do not block the sensing zone associated with the set of one or more sensors, as described elsewhere herein.Additionally, if the attachment 110 and the linkage 122 are in a position that is not an element of the permissible set of one or more positions, the controller 114 can determine that the attachment 110 and the linkage 122 are blocking the sensing zone assigned to the set of one or more sensors as described elsewhere herein.

[0042] As stated above, the Fig. 5A to 5B are provided as examples. Other examples may differ from what is shown in connection with Fig. 5A to 5B are described. Fig. Figure 6 is a representation of exemplary components of a device 600 associated with a perception sensor system of a machine. The device 600 can correspond to the perception sensor system 112, the controller 114, the plurality of sensors 126, the first set of one or more sensors 202, and / or the second set of one or more sensors 204. The perception sensor system 112, the controller 114, the plurality of sensors 126, the first set of one or more sensors 202, and / or the second set of one or more sensors 204 can include one or more devices 600 and / or one or more components of the device 600. As shown in Figure 6, the following applies: Fig. As shown in Figure 6, the device 600 can include a bus 610, a processor 620, a memory 630, an input component 640, an output component 650 and / or a communication component 660.

[0043] The bus 610 can comprise one or more components that enable wired and / or wireless communication between the components of the device 600. The bus 610 can connect two or more components of the Fig. 6. couple with each other, such as by operational coupling, communicative coupling, electronic coupling, and / or electrical coupling. For example, the bus 610 may include an electrical connection (e.g., a wire, a trace, and / or a conductor) and / or a wireless bus. The processor 620 may include a central processing unit, a graphics processing unit, a microprocessor, a control unit, a microcontroller, a digital signal processor, a general-purpose integrated circuit, an application-specific integrated circuit, or some other type of processing component. The processor 620 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 620 may include one or more processors that can be programmed to perform one or more operations or processes described elsewhere herein.

[0044] Memory 630 can include volatile and / or non-volatile memory. For example, memory 630 can include random-access memory (RAM), read-only memory (ROM), a hard disk drive, and / or another type of memory (e.g., flash memory, magnetic memory, and / or optical memory). Memory 630 can include internal memory (e.g., RAM, ROM, or a hard disk drive) and / or removable memory (e.g., removable via a universal serial bus connection). Memory 630 can be a non-volatile, computer-readable medium. Memory 630 can store information, one or more instructions, and / or software (e.g., one or more software applications) relating to the operation of the device 600. Memory 630 can include one or more memories that are coupled, for example, via bus 610 to the one or more processors (e.g., processor 620).are communicatively coupled). The communicative coupling between a processor 620 and a memory 630 enables the processor 620 to read and / or process information stored in the memory 630 and / or to store information in the memory 630.

[0045] The input component 640 enables the device 600 to receive inputs, such as user input and / or captured input. For example, the input component 640 can include a touchscreen, keyboard, keypad, mouse, button, microphone, switch, sensor, global positioning system sensor, global satellite navigation system sensor, accelerometer, gyroscope, and / or actuator. The output component 650 enables the device 600 to provide outputs, such as via a display, speaker, and / or light-emitting diode. The communication component 660 enables the device 600 to communicate with other devices via a wired and / or wireless connection.For example, the 660 communication component can include a receiver, a transmitter, a transceiver, a modem, a network interface card and / or an antenna.

[0046] The device 600 can perform one or more operations or processes described herein. For example, a non-volatile, computer-readable medium (e.g., memory 630) can store a set of instructions (e.g., one or more instructions or codes) for execution by the processor 620. The processor 620 can execute the set of instructions to perform one or more operations or processes described herein. The execution of the set of instructions by one or more processors 620 can cause the one or more processors 620 and / or the device 600 to perform one or more operations or processes described herein. Hardwired circuits can be used instead of, or in combination with, the instructions to perform one or more operations or processes described herein.Additionally or alternatively, the 620 processor can be configured to perform one or more of the operations or processes described herein. The implementations described herein are not limited to any particular combination of hardware circuitry and software.

[0047] The number and arrangement of the in Fig. The components shown in the 6 illustrations are provided as an example. The device 600 can have additional components, fewer components, different components, or components arranged differently than those shown. Fig. 6 shown. A set of components (e.g., one or more components) of the device 600 can perform one or more functions that are described as being performed by another set of components of the device 600. Commercial applicability

[0048] The implementations described herein can be used with any machine that includes a controller and a perception sensor system with multiple sets of one or more sensors. For example, the disclosed controller (e.g., controller 114) and the disclosed perception sensor system (e.g., perception sensor system 112) can be used with any machine that uses an attachment and linkage, such as a wheel loader, which has an attachment and linkage for loading, transporting, and unloading material (e.g., into a discharge point).

[0049] A machine may incorporate multiple sensors to detect the distance between the machine and a target (e.g., a discharge point, such as a dump truck) to facilitate an automated control operation (e.g., one that controls the movement of the machine or an attachment and linkage relative to the target). However, in many cases, the geometry of the attachment and linkage may obstruct the field of view of at least one of the multiple sensors. This at least one sensor detects and transmits erroneous data to the machine's control system, reducing the likelihood that the control system accurately determines the distance to the target. This can lead to the machine unintentionally making contact with the target, for example, due to an incorrect calculation of the stopping distance or the attachment's position relative to the target. This, in turn, causes damage (e.g.,Dents, cracks or other types of structural deformation) on the attachment and linkage, the machine and the target, which affects the performance of the attachment and linkage and reduces the service life of the attachment and linkage, the machine and the target.

[0050] In some implementations described herein, a machine controller can cause (e.g., selectively cause) perceptual data acquired by a set of one or more sensors of the machine's perceptual sensor system to be used or not used in conjunction with an automatic control operation. If the machine is a wheel loader or other similar machine that includes an attachment and linkage (e.g., for loading, transporting, and unloading material, such as into a discharge destination), the automatic control operation can be associated with the machine (e.g., to cause the machine to move to or from the discharge destination) and / or with the attachment and linkage (e.g., to cause the attachment and linkage to move into specific positions based on the machine's proximity to the discharge destination).

[0051] For example, the controller can determine that the implement and linkage are in a specific position and can then, based on that position, determine whether the implement and linkage are blocking sensing zones of the sensing sensor system (where each sensing zone is associated with a set of one or more sensors in the sensing sensor system). If the implement and linkage are blocking a sensing zone, the controller causes sensing data acquired by a set of one or more sensors associated with the sensing zone to be used in conjunction with the automatic control operation. Alternatively, if the implement and linkage are blocking the sensing zone, the controller causes the sensing data not to be used in conjunction with the automatic control operation.

[0052] In this way, the control system prevents the automatic steering process from using faulty sensor data (e.g., data from one or more sensors obstructed by the implement and boom). This makes it more likely that the control system, in conjunction with the automatic steering process, can accurately determine the distance to a target (e.g., a delivery point). This reduces the likelihood of the machine unintentionally coming into contact with the target, for example, due to an incorrect calculation of the stopping distance or the position of the implement and boom relative to the target. Consequently, in many cases, damage (e.g., dents, cracks, or other types of structural deformation) to the implement and boom, the machine, and the target is prevented, thus improving the performance and extending the service life of the implement and boom, the machine, and the target.

[0053] Furthermore, in some implementations, the controller initiates an automatic perception zone operation to identify a perception zone of the perception sensor system associated with a set of one or more sensors. The perception zone is associated with a permissible set of one or more attachment and linkage positions, which includes attachment and linkage positions that do not, or only minimally, obstruct the field of view of the set of one or more sensors. The controller may cause the attachment and linkage to move across a range of positions to identify the permissible set of one or more positions and thus identify the perception zone.

[0054] In this way, the controller dynamically determines a perception zone of the perception sensor system, which is assigned to a set of one or more sensors, based on the actual movement of the implement and linkage over a range of positions, rather than on configuration information assigned to the set of one or more sensors. This increases the likelihood that the perception zone will be accurately identified given the real-world design and operation of the implement and linkage, as well as the set of one or more sensors. By assigning the perception zone to permissible positions of the implement and linkage, the controller only needs to process position data associated with the implement and linkage to determine whether the implement and linkage are blocking the perception zone, thus conserving computing resources (e.g., CPU, CPU, etc.).(including power resources, storage resources, processing resources) that would otherwise be used (e.g. by the controller) to analyze perception data captured by the set of one or more sensors to determine whether the attachment and linkage block a field of view of the set of one or more sensors.

Claims

[1] Machine (100), comprising: an attachment (110) and a linkage (122) which are assigned to a front (124) of the machine (100); a perception sensor system (112) comprising a first set of one or more sensors (202) and a second set of one or more sensors (204) assigned to the front (124) of the machine (100); and a controller (114) designed for a: Identify that the attachment (110) and the linkage (122) are in a first position; Determine, based on identifying that the implement (110) and the linkage (122) are in the first position, that the implement (110) and the linkage (122) do not block a first sensing zone (206) associated with the first set of one or more sensors (202); and Based on the determination that the implement (110) and the linkage (122) do not block the first sensing zone (206), cause the first sensing data acquired by the first set of one or more sensors (202) to be used in conjunction with an automatic control operation assigned to at least one of the machine (100) or the implement (110) and the linkage (122). [2] Machine (100) according to claim 1, wherein the control (114) is further designed for: Identify, after instigating that the initial perception data are used in conjunction with the automatic control process, that the implement (110) and the linkage (122) are in a second position; Determine, based on identifying that the implement (110) and the linkage (122) are in the second position, that the implement (110) and the linkage (122) do not block a second sensing zone (208) associated with the second set of one or more sensors (204); and Based on the determination that the implement (110) and the linkage (122) do not block the second sensing zone (208), cause second sensing data acquired by the second set of one or more sensors (204) to be used in conjunction with an automatic control operation. [3] Machine (100) according to one of claims 1-2, wherein the control (114) is further designed for a: Determine, based on identifying that the implement (110) and the linkage (122) are in the first position, that the implement (110) and the linkage (122) do not block a second sensing zone (208) associated with the second set of one or more sensors (204); and Based on the determination that the implement (110) and the linkage (122) do not block the second sensing zone (208), cause second sensing data acquired by the second set of one or more sensors (204) to be used in conjunction with an automatic control operation. [4] Machine (100) according to one of claims 1-3, wherein the control (114) is further designed for: Determine, based on the first position of the implement (110) and the linkage (122), that the implement (110) and the linkage (122) block a second sensing zone (208) which is assigned to the second set of one or more sensors (204); and Based on the determination that the implement (110) and the linkage (122) block the second perception zone (208), cause second perception data acquired by the second set of one or more sensors (204) not to be used in conjunction with the automatic control operation. [5] Machine (100) according to any one of claims 1-4, wherein: the first set of one or more sensors (202) and the second set of one or more sensors (204) are arranged at different heights. [6] Machine (100) according to any one of claims 1-5, wherein the control (114) is further designed for: Before identifying that the attachment (110) and the linkage (122) are in the first position, initiate an automatic sensing zone calibration procedure. [7] Machine (100) according to one of claims 1-6, wherein the control (114) to initiate the automatic perception zone calibration process is designed for a: Identifying a field of view of the first set of one or more sensors (202); Causing the attachment (110) and the linkage (122) to move over a range of positions; Determine, for each position in the range of positions, whether the attachment (110) and the linkage (122) obstruct the field of vision; Identifying a set of one or more positions within the range of positions associated with the attachment (110) and the linkage (122) that do not obstruct the field of vision; and Identifying the first perception zone (206) as being assigned to the set of one or more positions. [8] Control (114) of a machine (100), comprising: one or more storage devices; and one or more processors (620) coupled with one or more memories and configured for a: Identifying a field of view of a first set of one or more sensors (202) of a perception sensor system (112) of the machine (100); Causing an attachment (110) and a linkage (122) of the machine (100) to move over a range of positions; Determine, for each position in the range of positions, whether the attachment (110) and the linkage (122) obstruct the field of vision; Identifying a set of one or more positions within the range of positions associated with the attachment (110) and the linkage (122) that do not obstruct the field of vision; and Identifying a first perception zone (206) of the perception sensor system (112) as being assigned to the set of one or more positions. [9] Controller (114) according to claim 8, wherein the one or more processors (620) are further configured for: Identify that the attachment (110) and the linkage (122) are in a first position; Determine, based on identifying that the implement (110) and the linkage (122) are in the first position, that the implement (110) and the linkage (122) do not block the first sensing zone (206); and Based on the determination that the implement (110) and the linkage (122) do not block the first sensing zone (206), cause the first sensing data acquired by the first set of one or more sensors (202) to be used in conjunction with an automatic control operation assigned to at least one of the machine (100) or the implement (110) and the linkage (122). [10] Controller (114) according to one of claims 8-9, wherein the one or more processors (620) are further configured for: Identify, after instigating that the initial perception data are used in conjunction with the automatic control process, that the implement (110) and the linkage (122) are in a second position; Determine, based on identifying that the implement (110) and the linkage (122) are in the second position, that the implement (110) and the linkage (122) do not block a second sensing zone (208) of the sensing sensor system (112) associated with the second set of one or more sensors (204); and Based on the determination that the implement (110) and the linkage (122) do not block the second sensing zone (208), cause second sensing data acquired by the second set of one or more sensors (204) to be used in conjunction with an automatic control operation.

Images