Photographing function control system and photographing function control method
By using a camera-based control system to limit and control the zoom ratio when remotely operating a hydraulic excavator, the problem of remote operators having difficulty accurately grasping the working conditions is solved, enabling efficient environmental monitoring and operation under different conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- KOBELCO CONSTR MASCH CO LTD
- Filing Date
- 2022-03-14
- Publication Date
- 2026-05-29
AI Technical Summary
When remotely operating a hydraulic excavator, the operator may find it difficult to accurately grasp the working conditions, especially when the zoom ratio has not been restored to the minimum display ratio, resulting in insufficient confirmation of the surrounding environment and affecting work efficiency.
The system employs a shooting function control system. The first shooting function control element limits the zoom ratio when the machine is operable, and controls the zoom ratio according to the zoom operation when it is not operable. Combined with the shake correction function, it ensures that the operator can effectively grasp the working environment in different states.
With remote operation, operators can have an overview of the working environment, avoiding image delays and improving work efficiency and operational accuracy, especially reducing operator interference during temporary interruptions in the work process.
Smart Images
Figure CN115118873B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a technology for assisting operators in remotely operating hydraulic excavators and other construction machinery. Background Technology
[0002] One proposed technique involves an operator using a remote control device to operate a machine while referring to a remote image showing the machine's operating status (e.g., see Patent Document 1).
[0003] Existing technical documents
[0004] Patent documents
[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-021395
[0006] Patent document 2 Japanese Patent Application Publication No. 2017-021517 Summary of the Invention
[0007] The problem that the invention aims to solve
[0008] Compared to being physically on the work equipment, it's difficult to monitor the work status when operating remotely. Therefore, it's desirable to be able to display magnified, detailed images of specific areas (ATT and / or the work object). Thus, the operator adjusts the camera's magnification and orientation to focus on specific areas and display detailed magnified images.
[0009] Furthermore, remote operators are required to have a more accurate grasp of their surroundings while performing tasks. Therefore, after confirming the work situation, the operator de-enhances the camera's magnification setting and restores it to the minimum (initial) magnification before proceeding to display an image on the display device with the widest possible field of view.
[0010] However, if the operator forgets to undo the increased camera magnification setting, or if the camera magnification is not fully restored to the minimum (initial) magnification, it will hinder the operator's understanding of the surrounding environment, resulting in insufficient understanding of the surrounding environment required for the operation.
[0011] As disclosed in Patent Document 2, if it is desired to display both the driving image and the attention image simultaneously, it is difficult for the operator to determine the position of the attention image within the driving image. Furthermore, the attention image cannot completely fill the screen, thus hindering the confirmation of the surrounding situation.
[0012] Therefore, the object of the present invention is to provide a system, etc., that allows an operator to understand the environment of the machine in an appropriate manner based on the state of the machine.
[0013] Methods for solving problems
[0014] The shooting function control system of the present invention is a system for controlling the shooting function of a real shooting device, wherein the shooting device is mounted on a working machine that operates according to operations input through an input interface, and is used to acquire captured images and output them to an output interface.
[0015] The shooting function control system is characterized by having:
[0016] In the first shooting function control element, when the working machine is in an operable state or in the state where the working machine is operating (i.e., in the first state), the first shooting function control element uses the zoom function of the shooting device to control the zoom ratio to below a specified magnification.
[0017] The second shooting function control element, when the working machine is in an inoperable state or the working machine is in a stopped state (i.e., the second state), utilizes the zoom function of the shooting device to control the zoom ratio based on the zoom operation input through the input interface.
[0018] According to the shooting function control system with this structure, in the first state, the operator can view the environment of the working machine from above by viewing the captured image output to the output interface, and operate the working machine through the input interface. On the other hand, in the second state, based on the zoom operation input by the operator through the input interface, the zoom function of the actual shooting device is activated, and a captured image with any display magnification is output through the output interface, allowing the operator to view the environment of the working machine in the desired manner. Attached Figure Description
[0019] Figure 1 This is an explanatory diagram of the structure of a remote operation assistance system.
[0020] Figure 2 This is an explanatory diagram of the structure of the remote operation device.
[0021] Figure 3 It is an explanatory diagram related to the structure of the operating machinery.
[0022] Figure 4 This is an explanatory diagram of the functions of the remote operation assistance system (shooting function control system).
[0023] Figure 5 It is an explanatory diagram of the working environment.
[0024] Figure 6 This is an explanatory diagram of the control method for the zoom function of a real-world shooting device.
[0025] Explanation of reference numerals in the attached figures
[0026] 10: Remote operation auxiliary server; 20: Remote operation device; 40: Working machinery; 102: Database; 121: First auxiliary processing element; 122: Second auxiliary processing element (first shooting function control element, second shooting function control element); 200: Remote control device; 210: Remote input interface; 211: Remote operation mechanism; 220: Remote output interface; 221: Remote image output device; 222: Remote audio output device; 400: Real machine control device; 41: Real machine input interface; 42: Real machine output interface; 424: Cab (driver's cab); 440: Working mechanism; 445: Bucket (working part); SW+: Zoom in switch; SW-: Zoom out switch. Detailed Implementation
[0027] (Structure of a remote operation assistance system)
[0028] Figure 1 The remote operation assistance system shown consists of a remote operation assistance server 10 and a remote operation device 20 for remotely operating the work machinery 40. The remote operation assistance server 10, the remote operation device 20, and the work machinery 40 are configured to communicate with each other via a network. The communication network between the remote operation assistance server 10 and the remote operation device 20, and the communication network between the remote operation assistance server 10 and the work machinery 40, can use the same communication network or different communication networks.
[0029] (Configuration of remote operation auxiliary server)
[0030] The remote operation assistance server 10 includes a database 102, a first auxiliary processing element 121, and a second auxiliary processing element 122. The database 102 stores and saves captured image data, etc. The database 102 may also be configured as a database server independent of the remote operation assistance server 10. Each auxiliary processing element is configured as a processing unit (a single-core processor or a multi-core processor, or a processor core constituting such a processor), reads necessary data and software from a storage device such as a memory, and performs computational processing according to the software, described later, on the object of that data. In this embodiment, the remote operation assistance server 10 constitutes a "capture function control system," and the second auxiliary processing element 122 constitutes a "first capture function control element" and a "second capture function control element."
[0031] (Structure of the remote operation device)
[0032] The remote operation device 20 includes a remote control device 200, a remote input interface 210, and a remote output interface 220. The remote control device 200 is composed of a processing unit (a single-core processor or a multi-core processor, or a processor core constituting the processor), which reads necessary data and software from storage devices such as memory, and performs arithmetic processing according to the software based on the data.
[0033] The remote input interface 210 includes a remote operation mechanism 211. The remote output interface 220 includes a remote image output device 221, a remote audio output device 222, and a remote wireless communication device 224.
[0034] The remote control mechanism 211 includes a travel control device, a slewing control device, a boom control device, a stick control device, and a bucket control device. Each control device has a lever for receiving rotational operations. The travel control device's lever (travel lever) is operated to move the lower traveling body 410 of the work machinery 40. The travel lever can also serve as a travel pedal. For example, a travel pedal fixed to the base or lower end of the travel lever can be provided. The slewing control device's lever (slewing lever) is operated to move the hydraulic slewing motor constituting the slewing mechanism 430 of the work machinery 40. The boom control device's lever (boom lever) is operated to actuate the boom cylinder 442 of the work machinery 40. The stick control device's lever (arm lever) is operated to actuate the stick cylinder 444 of the work machinery 40. The bucket control device's lever (bucket stick) is operated to actuate the bucket cylinder 446 of the work machinery 40.
[0035] like Figure 2 As shown, the levers constituting the remote operation mechanism 211 are arranged, for example, around a seat St for the operator to sit on. The seat St can be a high-backed chair with armrests, a low-backed chair without a headrest, or a chair without a backrest, or any other type of seating where the operator can sit.
[0036] A pair of left and right travel levers 2110 are arranged in front of the seat St, corresponding to the left and right tracks. One lever can function as multiple levers. For example, in Figure 2 The left-side operating lever 2111 on the front of the left side frame of the seat St, as shown, functions as an arm when operated in the forward / backward direction, and as a swivel lever when operated in the left / right direction. Similarly, when operating in the left / right direction... Figure 2The right-side operating lever 2112, located in front of the right-side frame of the seat St, functions as a boom lever when operated in the forward / backward direction and as a bucket lever when operated in the left / right direction. The lever mode can be changed arbitrarily according to the operator's instructions.
[0037] On the left operating lever 2111 and the right operating lever 2112, respectively, which are held by the operator, there are zoom in switch SW+ and zoom out switch SW-, which are used to adjust the zoom in angle and zoom out angle of the actual shooting device 412, as described later.
[0038] For example Figure 2 As shown, the remote image output device 221 consists of a central remote image output device 2210, a left remote image output device 2211, and a right remote image output device 2212, each with a roughly rectangular screen. The central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 are respectively positioned in front of, diagonally to the left front of, and diagonally to the right front of the seat St. The shape and size of the screen (image display area) of each of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 can be the same or different.
[0039] like Figure 2 As shown, the right edge of the left remote image output device 2211 is adjacent to the left edge of the central remote image output device 2210, with the screen of the central remote image output device 2210 and the screen of the left remote image output device 2211 forming a tilt angle θ1 (e.g., 120°≤θ1≤150°). Figure 2 As shown, the screens of the central remote image output device 2210 and the right remote image output device 2212 are tilted at an angle θ2 (e.g., 120°≤θ2≤150°), with the left edge of the right remote image output device 2212 adjacent to the right edge of the central remote image output device 2210. These tilt angles θ1 and θ2 can be the same or different angles.
[0040] The screens of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 can be parallel or tilted relative to the vertical direction. At least one of the image output devices 2210, 2211, and 2212 can also be composed of multiple image output devices. For example, the central remote image output device 2210 can also be composed of a pair of vertically adjacent image output devices with a generally rectangular screen shape.
[0041] The remote audio output device 222 consists of one or more speakers. For example, Figure 2 As shown, the remote audio output device 222 consists of a central audio output device 2220, a left audio output device 2221, and a right audio output device 2222, respectively located behind the seat St, behind the left armrest, and behind the right armrest. The specifications of the central audio output device 2220, the left audio output device 2221, and the right audio output device 2222 can be the same or different.
[0042] (Structure of the operating machinery)
[0043] The operating machinery 40 includes a machine control device 400, a machine input interface 41, a machine output interface 42, and a working mechanism 440. The machine control device 400 includes an image processing device 30. The image processing device 30 includes a status detection element 31, an image prediction element 32, and an image compression element 34. The components of the machine control device 400 and the image processing device 30 are respectively composed of a computing processing device (a single-core processor or a multi-core processor, or a processor core constituting the processor), which reads necessary data and software from storage devices such as memory, and performs calculations according to the software based on the data.
[0044] Construction machinery 40, for example, is a tracked excavator (construction machinery), such as... Figure 3 As shown, the device includes a tracked lower traveling body 410 and an upper rotating body 420 rotatably mounted on the lower traveling body 410 via a rotating mechanism 430. A driver's cab 424 is provided on the front left side of the upper rotating body 420. A working mechanism 440 is provided on the front center of the upper rotating body 420.
[0045] The machine input interface 41 includes a machine operation mechanism 411, a machine imaging device 412, and a positioning device 414. The machine operation mechanism 411 has multiple joysticks arranged similarly to the remote operation mechanism 211 around the seat located inside the cab 424. A drive mechanism or robot that receives signals corresponding to the operation mode of the remote joysticks and moves the machine joysticks based on these received signals is located in the cab 424. The machine imaging device 412 is located, for example, inside the cab 424, and captures images of the environment containing at least a portion of the operating mechanism 440 through the front window and a pair of side windows. This environment may omit some or all of the front and side windows. The positioning device 414 consists of a GPS device and, if necessary, a gyroscope sensor, etc.
[0046] The actual output interface 42 is equipped with actual wireless communication device 422.
[0047] like Figure 3 As shown, the working mechanism 440, which serves as the working mechanism, includes: a boom 441, which is mounted on the upper rotating body 420 for lifting and lowering; a stick 443, which is rotatably connected to the top of the boom 441; and a bucket 445, which is rotatably connected to the top of the stick 443. The working mechanism 440 is equipped with a boom cylinder 442, a stick cylinder 444, and a bucket cylinder 446, all of which are telescopic hydraulic cylinders.
[0048] Boom cylinder 442 is located between boom 441 and upper slewing body 420, allowing it to extend and retract by receiving a supply of working oil, thereby causing boom 441 to rotate in the lifting direction. Stick cylinder 444 is located between stick 443 and boom 441, allowing it to extend and retract by receiving a supply of working oil, thereby causing stick 443 to rotate about a horizontal axis relative to boom 441. Bucket cylinder 446 is located between bucket 445 and stick 443, allowing it to extend and retract by receiving a supply of working oil, thereby causing bucket 445 to rotate about a horizontal axis relative to stick 443.
[0049] (Function)
[0050] use Figure 4 The flowchart shown illustrates the functions of the remote operation assistance system and the shooting function control system with the above-described structure. In this flowchart, the "C●" block is a symbol used for simplification and refers to the transmission and / or reception of data, as well as a conditional branch that executes branch direction processing based on the transmission and / or reception of such data.
[0051] In the remote operation device 20, it is determined whether the specified operation was input by the operator via the remote input interface 210. Figure 4 / Step 210 (STEP 210)). "Specified operation" is, for example, an operation performed by the operator on the remote input interface 210 to specify the work machinery 40 to be remotely operated. If the determination result is negative ( Figure 4 / Step 210: No), the series of processes ends. On the other hand, if the determination result is positive ( Figure 4 / Step 210: Yes), send an environment confirmation request to the remote operation assistance server 10 via the remote wireless communication device 224. Figure 4 / Step 210).
[0052] In the remote operation assistance server 10, upon receiving an environmental confirmation request, the first auxiliary processing element 121 sends the environmental confirmation request to the operating machinery 40 corresponding to the environmental confirmation request. Figure 4 / C10).
[0053] In the operating machinery 40, when an environmental confirmation request is received via the actual machine wireless communication device 422 ( Figure 4 / C40), the actual control device 400 acquires captured images through the actual shooting device 412, and sends the image data (after image processing) to the remote operation auxiliary server 10 through the actual wireless communication device 422. Figure 4 / Step 410).
[0054] In the remote operation assistance server 10, when the first assistance processing element 121 receives the captured image data ( Figure 4 / C11), the second auxiliary processing element 122 sends environmental image data corresponding to the captured image to the remote operation device 20 ( / C11), Figure 4 / Step 110). Environmental image data is not only the captured image data itself, but also image data representing simulated environmental images generated based on the captured images.
[0055] In the remote operation device 20, when environmental image data is received via the remote wireless communication device 224 ( Figure 4 / C21), the remote control device 200 outputs the environmental image corresponding to the environmental image data to the remote image output device 221 (C21). Figure 4 / Step 212).
[0056] Therefore, for example Figure 5 As shown, by defining the window frame of the cab 424, an environmental image of the boom 441, stick 443 (which is part of the working mechanism 440), and a pile of rubble or sand (which is the object of the work of the bucket 445) that is reflected in front of the cab 424 is output to the remote image output device 221.
[0057] In the remote operation device 20, the remote control device 200 identifies the zoom operation mode of the remote operation mechanism 211, and sends a zoom command corresponding to the zoom operation mode to the remote operation auxiliary server 10 via the remote wireless communication device 224. Figure 4 / Step 214). Determine the zoom operation method based on the operating posture of the zoom magnification switch SW+ and zoom reduction switch SW- located on the respective gripping parts of the left operating lever 2111 and the right operating lever 2112 (refer to...). Figure 2 By continuously pressing the zoom magnification switch SW+, the zoom magnification γ of the actual shooting device 412 gradually increases or increases in stages. By continuously pressing the zoom reduction switch SW-, the zoom magnification γ of the actual shooting device 412 gradually decreases or decreases in stages.
[0058] In the remote operation auxiliary server 10, when a zoom command is received ( Figure 4 / C12), the second auxiliary processing element 122 determines the status of the working machine 40 based on the communication results with the working machine 40. Figure 4 / Step 120). "First state" refers to the state in which the machine tool 40 can be operated (or the state in which the machine tool 40 is operating). "Second state" refers to the state in which the machine tool 40 cannot be operated (or the state in which the machine tool 40 is stopped). For example, the machine tool 40 can be determined to be in the first state or the second state based on whether the operating cut-off lever of the actual machine operating mechanism 411 constituting the machine tool 40 is in a prescribed position. Alternatively, the machine tool 40 can be determined to be in the first state or the second state based on whether the remote hydraulic cut-off lever, which serves as a remote input interface and mimics a hydraulic cut-off lever, is in a prescribed position.
[0059] Under the condition that the operating machinery 40 is in the first state ( Figure 4 / Step 120…1), the second auxiliary processing element 122 (first shooting function control element) sends a first zoom command to the working machine 40 ( Figure 4 / Step 121). On the other hand, if it is determined that the operating machine 40 is in the second state ( Figure 4 / Step 120…2), through the second auxiliary processing element 122 (second shooting function control element), a second zoom command is sent to the working machine 40 ( Figure 4 / Step 122).
[0060] In the operating machinery 40, when a second zoom command is received via the actual machine wireless communication device 422 ( Figure 4 / C42), the actual camera control device 400 controls the zoom function of the actual shooting device 412 in a normal manner according to the operation mode of the zoom magnification switch SW+ and / or zoom reduction switch SW-, thereby adjusting its zoom ratio γ ( Figure 4 / Step 412).
[0061] For example, in zoom operation amount Figure 6 When the pattern shown by the solid line changes, it is as follows: Figure 6 As shown by the double-dotted line, the zoom ratio is adjusted proportionally to the amount of zoom operation (proportion factor = gain factor). The zoom operation increases when the zoom magnification switch SW+ is operated, and decreases when the zoom reduction switch SW- is operated. As a result, the width of the actual spatial area reflected in the captured image obtained by the actual shooting device 412, and subsequently in the environmental image output to the remote image output device 221, changes (see reference). Figure 4 Step 410 → … → Step 212 Figure 5 For example, when the zoom magnification γ increases due to the zoom magnification of the actual shooting device 412, the actual spatial area reflected in the ambient image becomes smaller or narrower than before. Conversely, when the zoom magnification γ decreases due to the zoom reduction of the actual shooting device 412, the actual spatial area reflected in the ambient image becomes larger or wider than before.
[0062] On the other hand, in the operating machinery 40, when the first zoom command is received via the actual machine wireless communication device 422 ( Figure 4 / C41), the camera control device 400 controls the zoom ratio of the camera shooting device 412 according to the operation of the zoom magnification switch SW+ and / or zoom reduction switch SW-, but in a restrictive manner compared to the usual method. Figure 4 / Step 411).
[0063] For example, in zoom operation amount Figure 6 When the pattern shown by the solid line changes, it is as follows: Figure 6 The method shown by the single-dotted line indicates that when the zoom operation amount is below the specified zoom magnification γ0, the zoom magnification is adjusted proportionally to that zoom operation amount. However, when the zoom operation amount is typically above the specified zoom magnification γ0 (during the period from time t1 to time t2), the zoom magnification γ is maintained at the specified zoom magnification γ0 regardless of changes in the zoom operation amount. Other examples include... Figure 6As shown by the dotted line, the zoom ratio γ of the actual shooting device 412 can also be controlled to a value below a specified magnification γ0 (e.g., a constant value). Furthermore, the zoom ratio of the actual shooting device 412 can be fixed at the minimum magnification or a magnification close to it, regardless of the operation mode of the zoom magnification switch SW+ and / or zoom reduction switch SW-.
[0064] In the remote operation device 20, the remote control device 200 identifies the operation mode of the remote operation mechanism 211, and the remote wireless communication device 224 sends a remote operation command corresponding to the operation mode to the remote operation auxiliary server 10. Figure 4 / Step 220).
[0065] In the remote operation auxiliary server 10, when the second auxiliary processing element 122 receives the remote operation command, the remote operation command is sent to the working machine 40 through the first auxiliary processing element 121. Figure 4 / C14).
[0066] In the operating machinery 40, when the actual machine control device 400 receives the operation command through the actual machine wireless communication device 422 ( Figure 4 / C44), controls the actions of the operating mechanism 440, etc. Figure 4 (Step 420). For example, performing an operation in which the bucket 445 scoops up the soil in front of the working machine 40 and dumps the soil from the bucket 445 after the upper rotating body 420 is rotated.
[0067] (Effects)
[0068] According to the shooting function control system of the remote operation assistance system having the above structure, when the working machine 40 is in the first state, the display magnification (zoom magnification γ) of the captured image (environmental image) output to the remote output interface 220 is controlled only when the display magnification is below a specified display magnification, or the display magnification is fixed to a value below a specified display magnification (see reference). Figure 4 / Step 120…1→Step 121→C41→Step 411、 Figure 6 (Single-dot lines and dashed lines). Thus, the operator can understand the environment of the machine 40 from above by viewing the captured images (environmental images) output to the remote output interface 220 (remote image output device 221), and operate the machine 40 through the remote input interface 210 (remote operating mechanism 211).
[0069] On the other hand, when the operating machinery 40 is in the second state, the zoom function of the actual shooting device 412 is utilized based on the operator's zoom operation via the remote input interface 210 (zoom magnification switch SW+ and zoom reduction switch SW-). Figure 4 Step 120…2→STEP122→C42→STEP412、 Figure 6 (Double-dotted lines). Thus, it is possible to output captured images (environmental images) at any display magnification via the remote output interface 220, thereby enabling the operator to monitor the environment of the operating machinery 40 in the manner desired.
[0070] (Other embodiments of the present invention)
[0071] In the above embodiment, the remote operation auxiliary server 10 constitutes the shooting function control system, and the second auxiliary processing element 122 constitutes the first shooting function control element and the second shooting function control element. However, as another embodiment, the shooting function control system may also be constituted by the operating machine 40 and / or the remote operation device 20, and the first shooting function control element and / or the second shooting function control element may be constituted by the actual machine control device 400 and / or the remote control device 200.
[0072] When the working machine 40 is in the second state, the zoom level γ of the shooting function of the actual shooting device 412 can also change according to the working progress of the working machine 40. For example, when the working machine 40 uses the working mechanism 440 to dig... Figure 5 When working on piles of rubble or sand (first state), if the operator wants to check the excavation face during the excavation process, the operation may be temporarily interrupted (second state) to perform a zoom operation via remote input interface 210 (zoom magnification switch SW+ and zoom reduction switch SW-). The zoom magnification γ can also be adjusted at a higher speed than usual during this zoom operation. If the excavation face deepens as the operation progresses, the speed at which the zoom magnification γ changes during the zoom operation can be further increased.
[0073] According to the shooting function control system with this structure, in the second state, the zoom magnification and / or zoom reduction speed is adjusted according to the different working progress of the machine, thus improving the operator's ease of understanding of the machine's environment. In particular, when the operation is interrupted to confirm the work object, the interruption time can be shortened, thereby improving work efficiency.
[0074] Alternatively, the second auxiliary processing element 122 (first shooting function control element) can send the "first shake correction command" together with the first zoom command to the operating machine 40 (see reference). Figure 4 / Step 121), the shake correction function of the actual shooting device 412 is stopped, and the second auxiliary processing element 122 (second shooting function control element) sends the "second shake correction command" and the second zoom command to the operating machine 40 (refer to step 121). Figure 4 / Step 122), shake correction command to perform the shooting function of the actual shooting device 412.
[0075] According to the shooting function control system with this structure, in the first state, by stopping the shake correction function of the actual shooting device 412, it is possible to prevent an increase in image processing costs for the captured image. Therefore, transmission delay from the actual shooting device 412 to the remote output interface 220 (remote image output device 221) is avoided, and consequently, output delay of the captured image (environmental image) on the remote output interface 220 is avoided (see reference). Figure 4 / Step 410→C11→Step 110→C21→Step 212、 Figure 5 As a result, it is possible to avoid situations where the operator has difficulty understanding the environment of the machine 40, and consequently, when operating the machine 40 through the remote input interface 210 (remote operating mechanism 211).
[0076] On the other hand, in the second state, by utilizing the shake correction function of the actual shooting device 412, the environment of the operating machinery 40 can be easily and accurately grasped through the shake-corrected captured image (environmental image) output to the remote output interface 220 (refer to...). Figure 5 ).
[0077] In the above embodiment, when the machine 40 is in the first state, the display magnification of the captured image (environmental image) output to the remote output interface 220 is controlled only within a range where the display magnification is below a specified display magnification. However, this control of limiting the display magnification can also be performed only after the machine 40 has just entered the first state. As a result, even when the machine 40 is in the first state, captured images (environmental images) at any display magnification can be output through the remote output interface 220, allowing the operator to grasp the environment of the machine 40 in the desired manner. Furthermore, after the machine 40 has just entered the first state, it is possible to avoid situations where the operator encounters obstacles in grasping the environment of the machine 40, and consequently in operating the machine 40 through the remote input interface 210 (remote operating mechanism 211).
[0078] In the shooting function control system having the above structure, it is preferable that, in the first state, the first shooting function control element stops the zoom function, which is the shooting function of the actual shooting device.
[0079] According to the shooting function control system with this structure, in the first state, the display magnification of the captured image output to the output interface is fixed to a magnification below a specified display magnification. Therefore, the operator can operate the machine through the input interface while having an overview of the machine's environment.
[0080] In the shooting function control system having the aforementioned structure, preferably, in the second state, the second shooting function control element causes the rate of change of the display magnification of the captured image in the zoom function of the shooting function of the actual shooting device to change according to the progress of the operation of the working machinery.
[0081] According to the shooting function control system with this structure, in the second state, the zoom magnification and / or zoom reduction speed is adjusted according to the different operating progress of the machine, thus improving the operator's convenience in understanding the environment of the machine.
[0082] In the shooting function control system having the aforementioned structure, preferably, in the first state, the first shooting function control element stops the shake correction function, which is the shooting function of the actual shooting device; and in the second state, the second shooting function control element performs the shake correction function, which is the shooting function of the actual shooting device.
[0083] According to the shooting function control system with this structure, in the first state, by stopping the shake correction function of the actual shooting device, the increased image processing cost of the captured image can be prevented. Therefore, transmission delay from the actual shooting device to the output interface is avoided, and consequently, output delay of the captured image at the output interface is avoided. As a result, situations where the operator encounters obstacles in understanding the environment of the working machinery, and consequently in operating the working machinery through the input interface, can be avoided. On the other hand, in the second state, by utilizing the shake correction function of the actual shooting device, the shake-corrected captured image output to the output interface allows the operator to easily and accurately understand the environment of the working machinery.
Claims
1. A shooting function control system, which is a system for controlling the shooting function of a real shooting device, wherein, The actual shooting device is mounted on the operating machinery that operates according to the operation input through the input interface, and is used to acquire the captured image and output it to the output interface. The shooting function control system is characterized by having: In the first shooting function control element, when the working machine is in an operable state or the working machine is in operation (i.e., in the first state), the first shooting function control element uses the zoom function of the shooting device to control the zoom ratio to below a specified magnification. The second shooting function control element, when the working machine is in an inoperable state or the working machine is in a stopped state (i.e., the second state), utilizes the zoom function of the shooting device to control the zoom ratio based on the zoom operation input through the input interface.
2. The shooting function control system according to claim 1, characterized in that, In the first state, the first shooting function control element stops the zoom function, which is the shooting function of the actual shooting device.
3. The shooting function control system according to claim 1, characterized in that, In the second state, the second shooting function control element changes the rate of change of the display magnification of the captured image in the zoom function of the shooting function of the actual shooting device according to the progress of the operation of the working machinery.
4. The shooting function control system according to claim 1, characterized in that, In the first state, the first shooting function control element stops the shake correction function, which is the shooting function of the actual shooting device; In the second state, the second shooting function control element performs the shake correction function as the shooting function of the actual shooting device.
5. A shooting function control method, wherein the shooting function control method is a method for controlling the shooting function of a real shooting device, wherein, The actual shooting device is mounted on the operating machinery that operates according to the operation input through the input interface, and is used to acquire the captured image and output it to the output interface. The shooting function control method is characterized in that it includes a first shooting control process and a second shooting control process as steps executed by a computing processing device. In the first shooting control process, when the working machine is in an operable state or in the state where the working machine is operating (i.e., the first state), the zoom function, which is the shooting function of the actual shooting device, is used to control the zoom ratio to below a specified ratio. In the second shooting control process, when the working machine is in an inoperable state or the working machine is in a stopped state, i.e., the second state, the zoom function, which is the shooting function of the actual shooting device, is used to control the zoom ratio according to the zoom operation input through the input interface.