Verification of perception information obtained by a perception sensor system comprising a plurality of sensors

Abstract

A controller can obtain perception information from a perception sensor system of a machine that includes a plurality of sensors, where the perception information includes respective perception data captured by the plurality of sensors. The controller can determine validity information associated with the perception information. The controller can select a portion of the perception information based on the validity information. The controller can determine a current distance to a target based on the portion of the perception information.

Description

Technical Field

[0001] This disclosure generally relates to a machine, and, for example, to the verification of perceived information obtained by a sensing sensor system comprising multiple sensors of the machine. Background Technology

[0002] To perform a dumping operation, a machine (such as a wheel loader) may use implements (e.g., a bucket or another implement) to load and transport materials (e.g., asphalt, debris, soil, snow, feed, gravel, logs, ore, recycled materials, rocks, sand, sawdust, or similar materials) and dump the material into a dumping target (e.g., another machine, such as a dump truck). The machine may include multiple sensors for detecting the distance from the machine to the dumping target (e.g., a representative distance among multiple individually detected distances), such as to automate implement lifting operations. For example, the machine may use multiple sensors to identify when it is sufficiently close to the dumping target and thus automatically raise the implement to the dumping height so that the machine can efficiently dump the material from the implement into the dumping target as soon as it reaches the target.

[0003] However, many factors can affect the sensor's ability to accurately detect various distances to the dumping target, such as environmental factors (e.g., factors related to weather conditions, lighting conditions, and / or site conditions), sensor factors (e.g., factors related to field of view capability, resolution capability, detection speed capability, ranging capability, calibration issues, sensor fouling issues, and / or interference with other sensors and / or other electrical equipment), target factors (e.g., factors related to the detectability of the dumping target's size, the geometry of the dumping target, fouling of the dumping target, and / or the material of the dumping target), and machine factors (e.g., factors related to vibration and / or movement of the machine and / or implements and the links connecting the implements to the machine, such as obstruction of the sensor's field of view due to the position of the implements and links). Therefore, in many cases, the machine uses inaccurate sensor readings from one or more sensors, which inhibits the machine's ability to accurately determine the distance to the dumping target. In many cases, this can lead to the machine unintentionally contacting the dumping target, such as due to miscalculations of the stopping distance or the implement's position relative to the dumping target. This can, in turn, damage the implements and links, the machine, and the dumping target (e.g., dents, cracks, or other types of structural deformation). This can affect the performance of the implements and links, the machine, and the dumping target, and shorten their operational life.

[0004] The controller disclosed herein addresses one or more of the problems described above and / or other problems in the art. Summary of the Invention

[0005] A machine includes: a implement and a linkage; a sensing sensor system including a plurality of sensors; a machine sensor system including a plurality of other sensors; and a controller configured to: identify a previously determined distance to a target; obtain machine information from the machine sensor system; determine the position of the implement and the linkage and the steering angle of the machine based on the machine information; obtain sensing information from the sensing sensor system, wherein the sensing information includes corresponding sensing data captured by the plurality of sensors; determine validity information associated with the sensing information based on at least one of the previously determined distance to the target, the position of the implement and the linkage, or the steering angle of the machine; select a portion of the sensing information based on the validity information; and determine the current distance to the target based on the portion of the sensing information.

[0006] A controller for a machine includes: one or more memories; and one or more processors coupled to the one or more memories and configured to: identify a previously determined distance to a target; obtain machine information from a machine sensor system of the machine; obtain perception information from a perception sensor system of the machine including multiple sensors; determine validity information associated with the perception information based on at least one of the previously determined distance to the target or the machine information; select a portion of the perception information based on the validity information; and determine the current distance to the target based on the portion of the perception information.

[0007] A method includes: obtaining perception information from a machine's perception sensor system, which includes multiple sensors, by a machine's controller, wherein the perception information includes corresponding perception data captured by the multiple sensors; determining validity information associated with the perception information by the controller; selecting a portion of the perception information based on the validity information by the controller; and determining the current distance to a target by the controller based on the portion of the perception information. Attached Figure Description

[0008] Figure 1 This is a diagram of the example machine described in this article.

[0009] Figure 2 This is a diagram of an example configuration of the front of the machine.

[0010] Figures 3A to 3B This is a diagram of an example implementation described herein.

[0011] Figures 4A to 4B This is a diagram of an example implementation described herein.

[0012] Figure 5This is a diagram of an example component of a device associated with the verification of perceived information obtained by a sensing sensor system comprising multiple sensors. Detailed Implementation

[0013] This disclosure relates to a controller for a machine (e.g., performing a dumping operation) and is applicable to any machine capable of loading and moving materials (e.g., from a first position to different second positions) and / or dumping materials (e.g., dumping into a dumping target). For example, the machine can be any machine performing operations associated with an industry such as mining, construction, farming, transportation, or any other industry. As some examples, the machine can be a vehicle, wheel loader, backhoe loader, cold planer, compactor, logging machine, forestry machinery, conveyor, harvester, excavator, industrial loader, articulated boom loader, material handler, automatic grader, pipelaying machine, road reclaimer, skid steer loader, tractor, bulldozer, tractor scraper, or other above-ground, underground, aerial, or water-based equipment.

[0014] Figure 1 This is a diagram of the example machine 100 described herein. For example, machine 100 may include a mobile machine (such as...) Figure 1 (The wheel loader shown) or any other type of mobile machine. In addition, machine 100 can be a manned machine or an unmanned machine, and / or can be fully autonomous, semi-autonomous or remotely operated.

[0015] As shown in the figure, machine 100 may have a frame 102 supporting operator station 104, power system 106, drive system 108, implements 110, sensing sensor system 112, and controller 114. Operator station 104 may include operator controls 116 for operating machine 100 via power system 106. In some examples, machine 100 may not include operator station 104 and / or operator controls 116 (e.g., machine 100 may be controlled via other means, such as a remote control system). Operator station 104 may be configured to define an internal compartment 118 in which operator controls 116 are housed.

[0016] Power system 106 is configured to supply power to machine 100. Power system 106 may be operatively arranged to receive control signals from operator controls 116 located in operator station 104 and / or from controller 114. Additionally or alternatively, power system 106 may be operatively arranged with drive system 108 and / or implement 110 to selectively operate drive system 108 and / or implement 110 according to control signals. Power system 106 may provide operating power for propulsion of drive system 108 and / or operation of implement 110. Power system 106 may include an engine, motor, electric drive, fuel cell, and / or another type of power system.

[0017] Drive system 108 may be operatively arranged with power system 106 to selectively propel machine 100 via control signals. Drive system 108 may include a plurality of ground engagement components (such as wheels 120, as shown), which are movably connected to frame 102 via shafts, drive shafts and / or other components. Drive system 108 may be provided as a tracked drive system, a wheeled drive system or any other type of drive system configured to propel machine 100.

[0018] The implement 110 can be operatively arranged with the power system 106 such that the implement 110 can be selectively moved by control signals transmitted from operator controls 116 and / or controller 114 to the power system 106. For example... Figure 1 As shown, the tool 110 can be coupled to the machine 100 via a link 122 (such as located at the front 124 of the machine 100). The tool 110 may also be referred to as an accessory, working tool, working implement and / or tool, and other examples. Figure 1 The implement 110 is depicted as a bucket as an example. Other embodiments may include any other suitable implement 110 for a variety of tasks, including, for example, bulldozing, brushing, compacting, grading, lifting, loading, hoeing and / or loosening, and other examples. Example implements 110 include stump grinders, trenchers, brooms, brush cutters, cold planers, lawn mowers, mulching machines, processors, shredders, rakes, saws, snowplows, snow blowers, tillers, winches, augers, blades, crushers / hammers, compactors, cutters, forklifts, grader bits and end-point bits, grapples, blades and / or rippers, and other examples. As described elsewhere herein, implement 110 may include one or more electrically powered components or parts.

[0019] The sensing sensor system 112 includes a plurality of sensors 126 that may be coupled to the machine 100, for example, at the front 124 of the machine 100. The plurality of sensors 126 may include sonar sensors, cameras, light detection and ranging (LIDAR) sensors and / or radio detection and ranging (RADAR) sensors, or another type of sensor for sensing the environment of the machine 100. That is, the plurality of sensors 126 may include at least one sensor configured to capture sensing data that may (e.g., by the controller 114) be used to determine at least one of the distance to a target (e.g., dumping a target) or the height of the target.

[0020] The controller 114 may include an electronic control module (ECM) or another computing device. The controller 114 may be configured to use sensing data captured by one or more sensors of the sensing sensor system 112 in association with automatic control operations (which are associated with at least one of machine 100 or implement 110 and linkage 122), to perform automatic sensing zone calibration operations (e.g., in association with the sensing sensor system 112), and / or to perform one or more other actions, as further described herein.

[0021] The rear portion 128 of machine 100 may include an engine and a transmission. The engine can be any type of engine suitable for using machine 100 to perform work, such as an internal combustion engine, diesel engine, gasoline engine, gas-fueled engine, etc. In other examples, machine 100 may include an alternative power system besides an engine, such as a motor (e.g., an electric motor), a battery-powered system, a fuel cell, or another type of power system. The transmission transfers power from the engine to drive system 108 and / or implement 110.

[0022] The machine may include a machine sensor system 130, which includes a plurality of other sensors 132 that can be housed within the machine 100. The plurality of other sensors 132 may include a positioning sensor (e.g., a Global Positioning System (GPS) sensor or a Local Positioning System sensor) configured to determine the physical position of the machine 100, a position sensor (e.g., a rotation sensor or another sensor) configured to detect the position of the implement 110 and the link 122, a speed sensor configured to determine the speed of the machine 100 (e.g., when traveling over a surface), a steering angle sensor configured to determine the steering angle of the machine 100, and / or one or more other sensors.

[0023] As indicated above, Figure 1 Provided as an example. Other examples may be provided in combination. Figure 1 The descriptions are different.

[0024] Figure 2 This is a diagram of an example configuration 200 of the front 124 of machine 100 (e.g., when the implement 110 and link 122 are extended to a “high” position, as further described herein). The sensing sensor system 112 may include multiple sensors 126 positioned at various locations (e.g., on machine 100 at the front 124 of machine 100). For example, as... Figure 2 As shown, one or more sensors 126 may be positioned at a first height associated with (e.g., aligned or nearly aligned) the top of the operator platform 104, and one or more sensors 126 may be positioned at a second height associated with (e.g., aligned or nearly aligned) the wheels 120.

[0025] As indicated above, Figure 2 Provided as an example. Other examples may be provided in combination. Figure 2 The descriptions are different.

[0026] Figures 3A to 3B This is a diagram of the example implementation 300 described herein. Figures 3A to 3B A side view of the machine 100 is shown when the implement 110 and the link 122 are in different positions.

[0027] like Figure 3A As shown, implement 110 and link 122 may be in a “low” position (e.g., when implement 110 is aligned or nearly aligned with wheel 120). As part of automated control operation (e.g., associated with machine 100 or at least one of implement 110 and link 122), controller 114 may position implement 110 and link 122 in the low position. For example, as part of automated control operation, controller 114 may position implement 110 and link 122 in the low position (e.g., when machine 100 is traveling, such as in a forward direction) to enable loading and / or transporting materials, such as from a first position to a second position associated with a dumping target (e.g., another machine, such as a dump truck). Figure 3A As shown, when in a low position, the implement 110 and the link 122 can not obstruct the field of view 302 of the first sensor 126 of the sensing sensor system 112, and can obstruct the field of view 304 of the second sensor 126 of the sensing sensor system 112.

[0028] like Figure 3BAs shown, implement 110 and link 122 can be in a "high" position (e.g., when implement 110 is aligned or nearly aligned with the top of operator platform 104). As part of automated control operation, controller 114 can position implement 110 and link 122 in the high position. For example, as part of automated control operation, when machine 100 is within a "discharge distance" of the dumping target (e.g., within a threshold distance), controller 114 can (e.g., when machine 100 is traveling, such as in the forward direction) position implement 110 and link 122 in the high position (e.g., to facilitate the imminent dumping of material carried by implement 110). Figure 3B As shown, when in a high position, the machine 110 and the connecting rod 122 can block the field of view 302 of the first sensor 126 of the sensing sensor system 112, but can not block the field of view 304 of the second sensor 126 of the sensing sensor system 112.

[0029] As indicated above, Figures 3A to 3B Provided as an example. Other examples may be provided in combination. Figures 3A to 3B The descriptions are different.

[0030] Figures 4A to 4B This is a diagram of the example implementation 400 described herein. Figures 4A to 4B This illustrates how the controller 114 determines the current distance to the target, such as based on verification of perception information obtained by a perception sensor system 112 (e.g., including multiple sensors 126).

[0031] like Figure 4A As shown and as indicated by reference numeral 402, controller 114 can obtain machine information. For example, controller 114 can obtain machine information from machine sensor system 130 (e.g., from a plurality of other sensors 132 of machine sensor system 130). Machine information may include corresponding machine data captured by the plurality of other sensors 132. That is, each of the plurality of other sensors 132 may send the machine data captured by that other sensor 132 (e.g., in real time or near real time) to controller 114, and therefore controller 114 may collectively receive the corresponding machine data captured by the plurality of other sensors 132 as machine information.

[0032] As shown in reference numeral 404, controller 114 can determine the position of implement 110 and link 122. For example, when multiple other sensors 132 include position sensors, controller 114 can receive machine data indicating the position of implement 110 and link 122 from the position sensors (e.g., in real-time or near real-time). Controller 114 can process the machine data (e.g., parse and / or read, and other examples) to determine that implement 110 and link 122 are in the appropriate position. Implement 110 and link 122 may be in a position, for example, loading and / or transporting material (such as from a first position to a second position associated with a dumping target (e.g., another machine, such as a dump truck)). This position may be, for example, as described herein. Figures 3A to 3B The description may refer to a low position, a high position, or another position.

[0033] As shown in reference numeral 406, controller 114 can determine the steering angle of machine 100. For example, when multiple other sensors 132 include steering angle sensors, controller 114 can receive machine data indicating the steering angle of machine 100 from the steering angle sensors (e.g., in real time or near real time). Controller 114 can then process the machine data (e.g., parse and / or read, and other examples) to determine the steering angle of machine 100. Because the operator of machine 100 interacts with operator controls 116, machine 100 can have a steering angle to cause the machine (e.g., while traveling) to move in a specific direction (such as toward a target (e.g., dumping a target)).

[0034] like Figure 4B As shown and as indicated by reference numeral 408, controller 114 can identify a previously determined distance to the target. This previously determined distance may have already been determined by controller 114. For example, controller 114 may perform the actions described herein in a repetitive loop. Figures 4A to 4B Some or all of the operations described, and therefore the controller 114 may have previously determined the distance to the target when executing the previous loop (e.g., as described herein). Figure 4B (As further described). Therefore, after the previous cycle is completed, the controller 114 can recognize the distance as the previously determined distance to the target.

[0035] As shown in reference numeral 410, controller 114 can acquire sensing information. For example, controller 114 can acquire sensing information from sensing sensor system 112 (e.g., from multiple sensors 126 of sensing sensor system 112). The sensing information may include corresponding sensing data captured by the multiple sensors 126. That is, each of the multiple sensors 126 can send the sensing data captured by sensor 126 (e.g., in real time or near real time) to controller 114, and therefore controller 114 can collectively receive the corresponding sensing data captured by the multiple sensors 126 as sensing information.

[0036] As shown in reference numeral 412, controller 114 can determine validity information associated with the sensing information. Validity information can indicate for each of a plurality of sensors 126 whether the sensing data captured by sensor 126 (and included in the sensing information) is valid. That is, validity information can indicate whether the sensing data is accurate enough to serve as a basis for determining the current distance to a target (e.g., as further described herein).

[0037] The controller 114 may determine validity information based on at least one of a previously determined distance to the target or machine information. In some embodiments, the controller 114 may determine validity information based on at least one of a previously determined distance to the target, the position of the implement 110 and link 122, or the steering angle of the machine 100.

[0038] As an example, to determine validity information, controller 114 may identify sensing data captured by one of the multiple sensors 126 (e.g., a specific sensor 126) in the sensing information. Therefore, controller 114 may (e.g., based on the sensing data) determine the sensing distance to the target. For example, controller 114 may process the sensing data (e.g., parse and / or read, and other examples) to determine the sensing distance to the target (e.g., the distance sensed by the sensor 126 that captured the sensing data). Therefore, controller 114 may determine distance difference information associated with the sensing distance to the target.

[0039] To determine distance difference information, as an example, controller 114 may determine a distance difference (e.g., a first distance difference) between the perceived distance and a previously determined distance to the target (e.g., a distance identified by controller 114, as described herein with respect to reference 408). As another example, to determine distance difference information, controller 114 may identify a previously determined perceived distance to the target (e.g., a distance determined based on previously captured perception data by sensor 126) and may determine a distance difference (e.g., a second distance difference) between the perceived distance and the previously determined perceived distance. In an additional example, to determine distance difference information, controller 114 may determine one or more other perceived distances to the target based on other perception data captured by one or more other sensors 126 of the plurality of sensors 126, and may determine a corresponding distance difference (e.g., a corresponding third distance difference) between the perceived distance and one or more other perceived distances. Thus, the distance information may indicate at least one of a first distance difference, a second distance difference, or a corresponding third distance difference.

[0040] The controller 114 can then determine whether the sensing data is valid based on the distance difference information. For example, the controller 114 can determine whether the sensing data is valid based on at least one of a first distance difference, a second distance difference, or a corresponding third distance difference. The controller 114 can determine that the first distance difference, the second distance difference, and / or the corresponding third distance difference satisfy (e.g., less than) a distance difference threshold to determine that the sensing data is valid. That is, the controller 114 can determine that the sensing data is valid based on the following conditions: the sensing distance is sufficiently similar to a previously determined distance to the target (e.g., represented by the first distance difference), indicating that the sensing distance is likely accurate; the sensing distance is sufficiently similar to a previously determined sensing distance to the target (e.g., represented by the second distance difference), indicating that the sensing distance is likely accurate; and / or the sensing distance is sufficiently similar to one or more other sensing distances to the target (e.g., represented by the corresponding third distance), indicating that the sensing distance is likely accurate. Therefore, the controller 114 can make the validity information indicate that the sensing data is valid. Alternatively, controller 114 may determine that at least one of a first distance difference, a second distance difference, or a corresponding third distance difference does not meet (e.g., greater than or equal to) a distance difference threshold (or different difference thresholds) to determine that the perceived data is invalid. That is, controller 114 may determine that the perceived data is invalid based on the following conditions: the perceived distance is not sufficiently similar to a previously determined distance to the target (e.g., represented by a first distance difference), indicating that the perceived distance may be inaccurate; the perceived distance is not sufficiently similar to a previously determined perceived distance to the target (e.g., represented by a second distance difference), indicating that the perceived distance may be inaccurate; and / or the perceived distance is not sufficiently similar to one or more other perceived distances to the target (e.g., represented by a corresponding third distance), indicating that the perceived distance may be inaccurate. Therefore, controller 114 may cause validity information to indicate that the perceived data is invalid.

[0041] In another example, to determine validity information, controller 114 may identify perceived data captured by one of a plurality of sensors 126 (e.g., a specific sensor 126) in the perceived information. Controller 114 may also determine whether the field of view of sensor 126 is obstructed. For example, controller 114 may determine whether the field of view of sensor 126 is obstructed by the machine 110 and link 122 based on the position of the implement 110 and link 122 (e.g., as indicated by machine information). As a specific example, see reference... Figure 3AWhen the implement 110 and the link 122 are in a low position, the implement 110 and the link 122 may not obstruct the field of view 302 of the first sensor 126 of the sensing sensor system 112, but may obstruct the field of view 304 of the second sensor 126 of the sensing sensor system 112. Therefore, the controller 114 can determine, based on the low position of the implement 110 and the link 122, that the field of view 302 of the first sensor 126 is obstructed and the field of view 304 of the second sensor 126 is not obstructed.

[0042] Therefore, controller 114 can determine whether the sensing data is valid (e.g., based on whether the field of view of sensor 126 is obstructed). Controller 114 can determine that the sensing data is valid based on the determination that the field of view of sensor 126 is not obstructed. That is, controller 114 can determine that sensor 126 has an open detection field, which indicates that the sensing distance is likely accurate. Therefore, controller 114 can enable validity information to indicate that the sensing data is valid. Alternatively, controller 114 can determine that the sensing data is invalid based on the determination that the field of view of sensor 126 is obstructed. That is, controller 114 can determine that sensor 126 does not have an open detection field, which indicates that the sensing distance may be inaccurate. Therefore, controller 114 can enable validity information to indicate that the sensing data is invalid.

[0043] In an additional example, to determine validity information, controller 114 may identify perceived data captured by one of the plurality of sensors 126 (e.g., a specific sensor 126) in the perceived information. Controller 114 may also identify a first moment associated with a previously determined distance (e.g., as described herein with respect to reference 408), such as the moment when determining or completing the determination of the previously determined distance, and may identify a second moment associated with the perceived data, such as the moment when sensor 126 captures the perceived data or the moment when controller 114 obtains the perceived data. Therefore, controller 114 may determine whether the perceived data is valid based on the first and second moments. For example, controller 114 may determine whether the second moment occurs after the first moment. That is, controller 114 may determine whether sensor 126 captured or whether controller 114 obtained the perceived data after determining the previously determined distance, which may indicate whether the perceived data is new or old.

[0044] Therefore, controller 114 can determine whether the sensed data is valid (e.g., based on whether the second moment occurs after the first moment). Controller 114 can determine that the sensed data is valid based on the determination that the second moment occurs after the first moment. That is, controller 114 can determine that the sensed data is new, which indicates that the sensed distance may be accurate. Therefore, controller 114 can enable validity information to indicate that the sensed data is valid. Alternatively, controller 114 can determine that the sensed data is invalid based on the determination that the second moment does not occur after the first moment. That is, controller 114 can determine that the sensed data is old, which indicates that the sensed distance may be inaccurate. Therefore, controller 114 can enable validity information to indicate that the sensed data is invalid.

[0045] In another example, to determine validity information, controller 114 may identify perceived data captured by one of the multiple sensors 126 (e.g., a specific sensor 126) within the perceived information. Controller 114 may also determine whether the perceived data is associated with the planned path of machine 100. For example, controller 114 may base its determination on the steering angle of machine 100 (e.g., as described herein). Figure 4A (As described in reference numeral 406) the controller 114 determines the planned path of machine 100 and can determine whether the planned path is within the field of view of sensor 126. The controller 114 can determine that the perceived data is associated with the planned path based on the determination that the planned path is within the field of view of sensor 126, and alternatively, can determine that the perceived data is not associated with the planned path based on the determination that the planned path is not within the field of view of sensor 126.

[0046] Therefore, controller 114 can determine whether the sensed data is valid (e.g., based on whether the sensed data is associated with the planned path). Controller 114 can determine that the sensed data is valid based on whether it is associated with the planned path. That is, controller 114 can determine that the sensed data is associated with the planned path, which indicates that the sensed distance is likely accurate. Therefore, controller 114 can make the validity information indicate that the sensed data is valid. Alternatively, controller 114 can determine that the sensed data is invalid based on whether it is associated with the planned path. That is, controller 114 can determine that the sensed data is not associated with the planned path, which indicates that the sensed distance may be inaccurate. Therefore, controller 114 can make the validity information indicate that the sensed data is invalid.

[0047] like Figure 4BFurther as shown and as indicated by reference numeral 414, controller 114 may (e.g., based on validity information) select a portion of the perceived information. For example, controller 114 may identify a first portion of the perceived information including perceived data indicated as valid by the validity information, and may identify a second portion of the perceived information including perceived data indicated as invalid by the validity information. Therefore, controller 114 may select the first portion of the perceived information. That is, controller 114 may select a portion of the perceived information including (e.g., as indicated by the validity information) valid perceived data.

[0048] As indicated by reference numeral 416, controller 114 may determine the current distance to a target (e.g., based on this portion of the perceived information). For example, controller 114 may process this portion of the perceived information (e.g., using one or more analysis techniques, such as analysis techniques using Kalman filters or another time series analysis technique) to determine the current distance to the target. In some embodiments, this portion of the perceived information may include corresponding perceived data captured by at least some of the plurality of sensors 126. Thus, the current distance to the target may be a representative distance to the target of the plurality of sensors 126, wherein the representative distance is a fused distance (e.g., an average, or another type of integration or merging) of distances associated with valid perceived data captured by the plurality of sensors 126.

[0049] As indicated above, Figures 4A to 4B Provided as an example. Other examples may be provided in combination. Figures 4A to 4B The descriptions are different.

[0050] Figure 5 This is a diagram of example components of a device 500 associated with the verification of perceived information obtained by a sensing sensor system including multiple sensors. Device 500 may correspond to a sensing sensor system 112, a controller 114, multiple sensors 126, a machine sensor system 130, and / or multiple other sensors 132. The sensing sensor system 112, controller 114, multiple sensors 126, machine sensor system 130, and / or multiple other sensors 132 may include one or more devices 500 and / or one or more components of device 500. Figure 5 As shown, the device 500 may include a bus 510, a processor 520, a memory 530, an input unit 540, an output unit 550, and / or a communication unit 560.

[0051] Bus 510 may include one or more components that enable wired and / or wireless communication between components of device 500. Bus 510 may connect components such as via operative coupling, communicative coupling, electronic coupling, and / or electrical coupling. Figure 5Two or more components are coupled together. For example, bus 510 may include electrical connections (e.g., wires, traces, and / or leads) and / or a wireless bus. Processor 520 may include a central processing unit, graphics processing unit, microprocessor, controller, microcontroller, digital signal processor, field-programmable gate array, application-specific integrated circuit, and / or another type of processing unit. Processor 520 may be implemented in hardware, firmware, or a combination of hardware and software. Processor 520 may include one or more processors capable of being programmed to perform one or more operations or processes described elsewhere herein.

[0052] Memory 530 may include volatile memory and / or non-volatile memory. For example, memory 530 may include random access memory (RAM), read-only memory (ROM), hard disk drive, and / or another type of memory (e.g., flash memory, magnetic storage, and / or optical storage). Memory 530 may include internal memory (e.g., RAM, ROM, or hard disk drive) and / or removable memory (e.g., removable via a Universal Serial Bus connector). Memory 530 may be a non-transitory computer-readable medium. Memory 530 may store information related to the operation of device 500, one or more instructions, and / or software (e.g., one or more software applications). Memory 530 may include one or more memories, such as those coupled (e.g., communicatively coupled) to one or more processors (e.g., processor 520) via bus 510. The communicative coupling between processor 520 and memory 530 enables processor 520 to read and / or process information stored in memory 530 and / or store information in memory 530.

[0053] Input component 540 enables device 500 to receive input, such as user input and / or sensed input. For example, input component 540 may include a touchscreen, keyboard, keypad, mouse, button, microphone, switch, sensor, GPS sensor, GNSS sensor, accelerometer, gyroscope, and / or actuator. Output component 550 enables device 500 to provide output, such as via a display, speaker, and / or LED. Communication component 560 enables device 500 to communicate with other devices via wired and / or wireless connections. For example, communication component 560 may include a receiver, transmitter, transceiver, modem, network interface card, and / or antenna.

[0054] Device 500 may perform one or more operations or processes described herein. For example, a non-transitory computer-readable medium (e.g., memory 530) may store a set of instructions (e.g., one or more instructions or code) for execution by processor 520. Processor 520 may execute the set of instructions to perform one or more operations or processes described herein. Execution of the set of instructions by one or more processors 520 causes one or more processors 520 and / or device 500 to perform one or more operations or processes described herein. Hardwired circuitry may be used in place of or in combination with instructions to perform one or more operations or processes described herein. Additionally or alternatively, processor 520 may be configured to perform one or more operations or processes described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0055] Figure 5 The number and arrangement of components shown are provided as an example. Device 500 may include components related to... Figure 5 The components shown are compared to additional components, fewer components, different components, or components arranged in a different manner. The set of components of device 500 (e.g., one or more components) can perform one or more functions described as being performed by another set of components of device 500.

[0056] Industrial applicability

[0057] The implementation described herein can be used with any machine that includes a controller and a sensing sensor system that includes multiple sensors, such as any machine that utilizes implements and linkages, such as a wheel loader that includes implements and linkages to load, transport, and dump materials (e.g., dump them into a dumping target).

[0058] The machine may include multiple sensors for detecting the distance from the machine to a target (e.g., unloading a target, such as a dump truck), such as to facilitate automated control operations (e.g., controlling the movement of the machine relative to the target or the operation of the machine's implements and linkages). For example, multiple sensors may capture perceived information indicating various detected distances to the target and transmit it to the machine's controller, which then determines the distance to the target (e.g., based on a representative distance among the various detected distances). However, in many cases, the ability of one or more of the multiple sensors to accurately detect various distances to the target may be impaired due to environmental factors, sensor factors, target factors, and / or machine factors. Therefore, one or more sensors may capture erroneously detected distances and transmit them to the machine's controller, which (e.g., because the distance to the target is at least partially based on the erroneously detected distances) reduces the likelihood that the controller will determine the accurate distance to the target. This may lead to the machine inadvertently contacting the target, such as due to miscalculations of the stopping distance or the position of the machine's implements and linkages relative to the target. This, in turn, leads to damage to implements and links, machines, and targets (e.g., dents, cracks, or other types of structural deformation), which affects the performance of implements and links, machines, and targets and shortens their operational life.

[0059] In some implementations, the machine's controller may determine the current distance to the target based on verification of perceived information obtained by a sensing sensor system comprising multiple sensors of the machine. For example, the controller may obtain perceived information from the sensing system that includes corresponding perceived data captured by multiple sensors. The controller then determines verification information associated with the perceived information based on a previously determined distance to the target (e.g., a distance determined by the controller) and / or machine information (e.g., machine information indicating the positions of the machine's implements and links and / or the machine's steering angle). The verification information indicates for each of the multiple sensors whether the perceived data captured by that sensor is valid (e.g., whether it is accurate enough to serve as the basis for determining the current distance to the target).

[0060] To determine validity information, the controller can determine the perceived distance to the target (e.g., the distance sensed by the sensor that captured the perceived data) based on the perceived data captured by the sensor in the perceived information. The controller can determine that the perceived data is valid when the perceived distance to the target is sufficiently similar to a previously determined distance to the target, a previously determined perceived distance to the target (e.g., determined by the sensor), and / or one or more other perceived distances to the target (e.g., determined based on other perceived data captured by one or more other sensors in the perceived information). In some cases, the controller can determine the validity of the perceived data by (e.g., based on the position of the implement and linkage) determining that the sensor's field of view is not obstructed (e.g., not obstructed by the implement and linkage), by determining that the perceived data is new, and / or by (e.g., based on the machine's steering angle) determining that the perceived data is associated with the machine's planned path.

[0061] Therefore, the controller selects a portion of the perceived information (which includes validity information indicating valid perceived data) and determines the current distance to the target based on this portion of the perceived information (rather than based on a portion of the perceived information including validity information indicating invalid perceived data). For example, the controller processes this portion of the perceived information (e.g., using an analysis technique utilizing a Kalman filter or another analysis technique) to determine the current distance to the target. Thus, the current distance to the target is a representative distance to the target from multiple sensors, where the representative distance is the set of distances (e.g., an average or another type of set) associated with valid perceived data captured by multiple sensors.

[0062] In this way, the controller prevents the use of invalid (e.g., potentially inaccurate) sensing data, such as sensing data captured by faulty, obstructed, or incorrectly positioned sensors, when determining the current distance to the target. Therefore, the controller is more likely to accurately determine the current distance to the target by using only valid (e.g., potentially accurate) sensing data. This reduces the likelihood of the machine inadvertently contacting the target, such as due to miscalculations of stopping distance or the position of the implements and links relative to the target. Consequently, damage to implements and links, the machine, and the target (e.g., dents, cracks, or other types of structural deformation) is prevented in many cases, which improves the performance of implements and links, the machine, and the target and extends their operational life.

Claims

1. A machine, said machine comprising: Tools and connecting rods; A sensing sensor system, the sensing sensor system comprising multiple sensors; A machine sensor system, which includes multiple other sensors; and The controller is configured to: Identify the previously determined distance to the target; Obtain machine information from the machine's sensor system; The positions of the implement and the connecting rod, as well as the steering angle of the machine, are determined based on the machine information. Perception information is obtained from the perception sensor system. The sensing information includes corresponding sensing data captured by the plurality of sensors; The validity information associated with the perceived information is determined based on at least one of the previously determined distance to the target, the position of the implement and the linkage, or the steering angle of the machine; Select a portion of the perceived information based on the validity information; as well as The current distance to the target is determined based on a portion of the perceived information.

2. The machine of claim 1, wherein the validity information indicates for each of the plurality of sensors whether the sensing data captured by the sensor is valid.

3. The machine according to any one of claims 1 to 2, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; The sensing distance to the target is determined based on the sensing data; Determine the distance difference between the perceived distance and the previously determined distance; as well as The validity of the perceived data is determined based on the distance difference.

4. The machine according to any one of claims 1 to 3, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; The sensing distance to the target is determined based on the sensing data; Identify the previously determined sensing distance to the target, the previously determined sensing distance being determined based on sensing data previously captured by the sensor; Determine the distance difference between the perceived distance and the previously determined perceived distance; as well as The validity of the perceived data is determined based on the distance difference.

5. The machine according to any one of claims 1 to 4, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; The sensing distance to the target is determined based on the sensing data; One or more other perceived distances to the target are determined based on other perceived data captured by one or more other sensors among the plurality of sensors in the perceived information; Determine the corresponding distance difference between the sensing distance and one or more other sensing distances; as well as The validity of the perceived data is determined based on the corresponding distance difference.

6. The machine according to any one of claims 1 to 5, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; Based on the positions of the machine and the connecting rod, determine whether the sensor's field of view is obstructed by the machine and the connecting rod; as well as The validity of the sensing data is determined based on whether the sensor's field of view is obstructed.

7. The machine according to any one of claims 1 to 6, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; Identify the first moment associated with the previously determined distance; Identify the second moment associated with the perceived data; Determine whether the second moment occurs after the first moment; as well as The validity of the perceived data is determined based on whether the second moment occurs after the first moment.

8. The machine according to any one of claims 1 to 7, wherein, To determine the validity information, the controller is configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; The planned path of the machine is determined based on the machine's steering angle; Determine whether the perceived data is associated with the planned path; as well as The validity of the sensing data is determined by whether it is associated with the planned path.

9. A controller for a machine, the controller comprising: One or more memory units; and One or more processors, said one or more processors being coupled to said one or more memories and configured to: Identify the previously determined distance to the target; Machine information is obtained from the machine's sensor system; Perception information is obtained from the machine's perception sensor system, which includes multiple sensors; Validity information associated with the perceived information is determined based on at least one of the previously determined distance to the target or the machine information; Select a portion of the perceived information based on the validity information; as well as The current distance to the target is determined based on a portion of the perceived information.

10. The controller according to claim 9, wherein, To determine the validity information, the one or more processors are configured to: Identify the perceived data captured by one of the plurality of sensors in the perceived information; Based on the machine information, determine whether the perceived data is associated with the machine's planned path; as well as The validity of the sensing data is determined by whether it is associated with the planned path.

Images