Unmanned driving control system for snowbikes and method for controlling unmanned driving of snowbikes

Abstract

To provide comprehensive support for rescue operations during nighttime hours in snow-covered areas during winter. [Solution] When remotely controlling the snow bike's driving function by transmitting a snow bike control signal to a snow bike equipped with an FPV camera CAM, while viewing it on a display connected to a control device that receives video data transmitted from the FPV camera, the snow bike is equipped with an audio processing means that controls a microphone that collects human voices generated around the snow bike's body and a speaker that amplifies and outputs the sound transmitted from the control device, and the audio processing means simultaneously controls the microphone to transmit audio information collected by the microphone to the control device via the transmitting and receiving antenna, and controls the speaker to amplify the audio information received from the control device via the transmitting and receiving antenna.

Description

Technical Field

[0001] The present invention relates to an unmanned driving control system for a snowmobile that wirelessly controls a snowmobile traveling on a snow path to support rescue and ambulance activities on snow, and an unmanned driving control method for the unmanned driving control system of the snowmobile.

Background Art

[0002] Conventionally, a snowmobile has been put into practical use as a mobile vehicle traveling on snow.

[0003] Here, since it is not planned to drive on a public road when driving a snowmobile, there is no need to obtain a special driving license.

[0004] In addition, when driving a snowmobile on snow, the driver starts the engine, operates the accelerator and brake to move forward, and performs the basic operation of braking to stop the snowmobile, so it is utilized as a means of transportation in a limited snow area in a snow country.

[0005] On the other hand, mountaineers who go winter mountaineering may deviate from the climbing route and encounter a snow mountain accident if they do not respond to sudden changes in weather and climb while checking sufficient position information.

[0006] The following patent document is published as an example of a general snowmobile that runs on a gasoline engine.

[0007] In addition, in order to make a snowmobile unmanned, a technology for constantly grasping the position and posture of a snowmobile by combining GPS and IMU (inertial measurement unit) is published. Here, the unmanned snowmobile is equipped with means for detecting and avoiding surrounding obstacles using a visual sensor, laser scan, etc. In addition, as a communication system, a wireless communication system is provided for remote operation and data transmission.

[0008] Furthermore, when operating a snowmobile remotely, a controller is needed that automatically drives the snowmobile based on an autopilot algorithm, GPS data, and sensor information. Additionally, an efficient battery management system is required to support extended operation.

[0009] Furthermore, Patent Document 1 below discloses a technology relating to the driving control of a snowmobile, stating that "In order to provide a snowmobile having improved steering control, the snowmobile having improved steering control has a driving control system comprising an electric actuator coupled to a steering system having a user-operated steering element, the actuator having an electric actuator that applies torque to the steering system, a throttle, a plurality of sensors including a torque sensor and at least one additional sensor for generating terrain data and driving data, and at least one controller coupled to the actuator and sensors. The at least one controller selects a terrain condition mode using the generated terrain conditions and generated driving data, determines the torque to be applied in response to the angle and speed of rotation of the user-operated steering element, and activates the electric actuator to apply torque to the steering system, the torque being applied only by the electric actuator." [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Publication No. 2021-172337 [Overview of the project] [Problems that the invention aims to solve]

[0011] In order to quickly determine the location of the stranded climber and their bivouac site, and to transport rescue supplies, it is usually necessary to form a team of police officers and mountain rescue personnel, then split into multiple groups and move across the snow on foot. In particular, rescue operations are not carried out at night to prevent secondary accidents, and incidents have occurred where people have died because their bodies could not be sustained due to hypothermia.

[0012] Therefore, there is a strong need to develop a system that allows snowbikes to operate unmanned, even at night, to provide the fastest possible rescue for those in distress.

[0013] In particular, conventional technologies for autonomous snowbike operation do not integrate with camera technology. Especially for systems that operate autonomous snowbikes equipped with FPV cameras, the system needs to be designed according to the intended use, and it needs to be equipped with features that allow for the care of rescuers. A flexible system design is required that takes into account not only autonomous operation but also rescue operations.

[0014] The present invention was made to solve the above problems, and provides an unmanned driving control system for a snowbike and an unmanned driving control method for a snowbike, which enable a remote operator to control the starting and stopping of the snowbike's power supply, the direction of travel, the speed of travel, stopping of travel, and the turning on and off of the headlights, thereby enabling comprehensive support for rescue activities in snow-covered disaster areas in a mobile and rapid manner, such as transporting necessary supplies to climbers and residents isolated by natural disasters at night in snowy areas during winter, or transporting rescued persons on stretchers. [Means for solving the problem]

[0015] The unmanned driving control system for snowbikes of the present invention, which achieves the above objectives, has the following configuration.

[0016] The present invention relates to an unmanned snowbike driving control system that allows a snowbike equipped with an FPV camera to drive on snow unmanned, by transmitting a snowbike driving control signal to remotely control the driving function of the snowbike while viewing it on a display connected to a control device that receives video data transmitted from the FPV camera via a predetermined communication medium, wherein the snowbike is equipped with a microphone that collects human voices generated around the body of the snowbike and a speaker that amplifies and outputs sound transmitted from the control device, and the sound processing means controls the transmission of sound information collected by the microphone to the control device via the transmitting and receiving antenna and the amplification of sound information received from the control device via the transmitting and receiving antenna through the speaker. [Effects of the Invention]

[0017] According to the present invention, even at night in snow-covered areas during winter, it is possible to comprehensively support rescue activities in snow-covered disaster areas in a mobile and rapid manner by searching for climbers awaiting rescue, transporting necessary supplies to residents isolated by natural disasters, and transporting those in need on stretchers. [Brief explanation of the drawing]

[0018] The drawings illustrate specific embodiments of the present invention, including not only essential components of the invention but also optional and preferred embodiments. [Figure 1] A perspective view showing the exterior of the snowbike according to this embodiment. [Figure 2] A block diagram illustrating the configuration of the unmanned driving control system for the snowbike shown in this embodiment. [Figure 3] (a) is a block diagram illustrating the configuration of the first to fourth control devices connected to the central server shown in Figure 1, and (b) is a diagram illustrating the configuration of the program stored in the RAM shown in (a). [Figure 4] A block diagram illustrating the configuration of an unmanned snowbike drone remotely controlled by the first to fourth control devices shown in Figure 1. [Figure 5] A block diagram for explaining the configuration of the center server shown in FIG. 1. [Figure 6] A flowchart for explaining the remote control processing method of the unmanned driving control system of the snowmobile shown in this embodiment. [Figure 7] A flowchart for explaining the remote control processing method of the unmanned driving control system of the snowmobile shown in this embodiment. [Figure 8] A flowchart for explaining the remote control processing method of the unmanned driving control system of the snowmobile shown in this embodiment. [Figure 9] A flowchart for explaining the remote control processing method of the unmanned driving control system of the snowmobile shown in this embodiment. [Figure 10] A flowchart for explaining the remote control processing method of the unmanned driving control system of the snowmobile shown in this embodiment. [Figure 11] A schematic diagram showing an example of a travel guide icon displayed on the touch panel of the first to fourth control devices and the 3D goggles shown in FIG. 2.

Mode for Carrying Out the Invention

[0019] Next, the best mode for carrying out the present invention will be described with reference to the drawings.

[0020] <Description of System Configuration> 〔First Embodiment〕 FIG. 1 is a perspective view showing the appearance of a snowmobile to which the present invention is applied. In this embodiment, the traveling belt for traveling on snow is configured such that the rotational force is transmitted through a rotating gear driven by a gasoline engine, but it may also be configured by an electric motor or a hybrid of an electric motor and a gasoline engine. Also, the fuel of the engine may be hydrogen or oil extracted from plants.

[0021] In Figure 1, 101 is an air filter that purifies the air supplied to the engine. 102 is a battery that supplies 24V or 12V DC power. A fuse box (not shown) detects overcurrents entering the snowbike's electrical circuit and cuts off the power supply.

[0022] A V-belt (not shown) transmits the driving force to the engine drive gear, which rotates to propel the snowbike forward. 108 is the handle lever, which moves in a predetermined angle range either counterclockwise or clockwise, causing the ski section located at the front lower part to move in conjunction. 109 is the storage compartment, which is divided into a front storage compartment and a rear storage compartment on the rear side of the main body.

[0023] Furthermore, the rear storage compartment is configured to accommodate designated rescue equipment such as a thermal blanket, a foldable stretcher, bone fixation devices for fixing fractured bones, and joint fixation devices for fixing the cervical or lumbar vertebrae. Furthermore, the front storage compartment is equipped with a GPS transmitter that transmits location information to determine the rescue location. Furthermore, the front storage compartment is equipped with a front storage section for housing floodlights, emergency radio communication equipment, and batteries. Furthermore, the front storage compartment includes a forward storage section for arsenal flares and fireplace materials.

[0024] 112 is a rear carrier, configured to secure equipment. 114 is a taillight, which alerts snow bikes following behind that the vehicle is braking. 115 is a suspension, which absorbs vibrations caused by the vertical movement of the vehicle.

[0025] Furthermore, the front of the unit is equipped with a headlight, and an engagement member is provided for attaching and detaching a floodlight as a secondary light source. This makes it possible to spot rescuers 500-600m away, who would not be visible with the headlight alone. Furthermore, the rear or front of the vehicle body is equipped with a winch for towing a foldable stretcher used to transport rescued individuals.

[0026] Furthermore, the storage compartment 109 is equipped with a hot water tank for storing hot water at a predetermined temperature (40-42 degrees Celsius) to warm the injured area of ​​the rescued person, as well as a cardiopulmonary resuscitation kit (AED (Automated External Defibrillator)). This allows for rapid care of frostbite patients. Furthermore, the inside of the foldable stretcher is equipped with a designated heat source (powered by a lithium battery) to maintain the patient's body temperature at a predetermined level.

[0027] Furthermore, the folding stretcher is equipped with a temperature controller that allows for individual temperature setting of the temperature supplied from a predetermined heat source to the head side, upper body side, lower body side, and toe side of the stretcher.

[0028] 120 is an antenna that performs bidirectional communication with the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N using a predetermined communication protocol.

[0029] The CAM is an FPV camera that transmits captured video data via antenna 120 to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N.

[0030] Here, the operators and monitors who operate the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N remotely control the direction of travel, travel speed, stopping, and turning on / off the headlights of the snowbike drones SD1 to SD4 while viewing the video data of the snowbike drones SD1 to SD4 displayed on the monitor.

[0031] Figure 2 is a block diagram illustrating the configuration of the unmanned driving control system for a snowbike according to this embodiment.

[0032] In Figure 2, 1 is the central server, which coordinates communication with the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N. It also aggregates information from the first snowbike drone SD1, the second snowbike drone SD2, the third snowbike drone SD3, and the fourth snowbike drone SD4, which constitute the search and rescue team, via the network 21, and comprehensively controls data processing for sharing search and rescue activity information.

[0033] Here, the first control devices 3-1 to 3-N communicate bidirectionally with the first snowbike drone SD1, which is equipped with an FPV camera CAM1, via the network 21.

[0034] Similarly, the second control devices 4-1 to 4-N communicate bidirectionally with the second snowbike drone SD2, which is equipped with an FPV camera CAM2, via the network 21.

[0035] Similarly, the third control devices 5-1 to 5-N communicate bidirectionally with the third snowbike drone SD3, which is equipped with an FPV camera CAM3, via the network 21.

[0036] Similarly, the fourth control devices 2-1 to 2-N communicate bidirectionally with the fourth snowbike drone SD4, which is equipped with an FPV camera CAM4, via the network 21.

[0037] The first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N are configured to be connectable to data terminals equipped with screens that display video data captured by the FPV camera CAM mounted on the snowbike, or to goggle systems that display 3D images. The search spot areas will be described later.

[0038] D1-D4 are drones equipped with FPV cameras (CAM) and configured to fly over designated snow-covered road search areas using GPS signals, while simultaneously transmitting captured video data to the central server 1. SP1-SP4 are search spot areas, configured by the central server 1 located at the search team headquarters.

[0039] Figure 3 is a block diagram illustrating the configuration of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, which are connected to the central server 1 shown in Figure 1.

[0040] The first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N are envisioned to be devices such as a tablet, a PC, or a smartphone.

[0041] In Figure 3(a), 301 is the CPU, which executes various applications by loading the OS and control programs stored in ROM 302 into RAM 303 and running them. 304 is the communication unit, which controls communication for connecting to the center server 1 connected to network 21.

[0042] The display and keyboard may be configured as a single unit, a touch panel display 311A.

[0043] In the RAM 303 shown in Figure 3(b), the login unit 303-1 performs a process to authenticate the ID and password of the user performing the terminal operation.

[0044] 303-3 is a UI control unit that executes browser function processing and performs control to display on the touch panel displays 311A ​​provided by the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, from the platform 1A provided by the central server 1 shown in Figure 1.

[0045] Furthermore, the UI control unit 303-3 receives the aerial video file edited by the center server 1, saves it to a non-volatile memory (not shown) or a non-volatile memory area, and the video can be played by launching a video application downloaded by the download unit 303-2 from the application site.

[0046] 303-4 is the route selection unit, which executes the process of selecting the desired search route from the list of search routes displayed on the service screen provided by the central server 1.

[0047] The flight routes are configured to allow for the addition of optimal search routes based on multiple flight conditions, such as season, weather conditions, and the position of the sun, by AI analysis of aerial video data captured by drones D1 to D4.

[0048] 303-5 is the receiving unit, which receives AI-edited snow video data transmitted from one of the corresponding first snowbike drones SD1, second snowbike drone SD2, third snowbike drone SD3, or fourth snowbike drone SD4.

[0049] This allows for clearer image processing and visualization of the search area video data to be provided to operators of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, as well as to monitors assisting with the search. Specifically, by relaying the data to a data terminal (not shown) operated by a monitor assisting with the search, visually useful search information can be obtained at the front lines of rescue operations at the search site.

[0050] This allows for the rapid development of rescue operation plans and enables agile support for wide-area operations, from the rescue of those in need to their transport to hospitals.

[0051] The driving command unit 303-6 sets a rescue activity mode to support rescue activities for people lost in the snow when the first snowbike drone SD1, the second snowbike drone SD2, the third snowbike drone SD3, and the fourth snowbike drone SD4 are driving at night, controls the on / off switching of the lighting means, and controls the unmanned driving of the first snowbike drone SD1, the second snowbike drone SD2, the third snowbike drone SD3, and the fourth snowbike drone SD4 on the snow along a search route identified by terrain information pre-stored in the search video database 14B of the center server 1, while acquiring GPS signals.

[0052] Furthermore, the first control devices 3-1 to 3-N control the process of receiving video captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the first snowbike drone SD1 via antenna AT1. Antenna AT1 transmits driving control signals issued from the driving command unit 303-6 to the first snowbike drone SD1.

[0053] Furthermore, the video output unit 303-7 displays the video data captured by the FPV camera CAM, transmitted from the transmitting / receiving antenna AT of the first snowbike drone SD1, on the monitor screen of the touch panel display 311A ​​or the 3D goggles 311B.

[0054] The second control devices 4-1 to 4-N control the process of receiving video footage captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the second snowbike drone SD2 via antenna AT2. Antenna AT2 transmits driving control signals issued from the driving command unit 303-6 to the second snowbike drone SD2. The video output unit 303-7 displays the video data captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the second snowbike drone SD2 on the monitor screen of the touch panel display 311A ​​or the 3D goggles 311B.

[0055] The third control devices 5-1 to 5-N control the process of receiving video captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the third snowbike drone SD3 via antenna AT3. Antenna AT3 transmits driving control signals issued from the driving command unit 303-6 to the third snowbike drone SD3. The video output unit 303-7 displays the video data captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the third snowbike drone SD3 on the monitor screen of the touch panel display 311A ​​or the 3D goggles 311B.

[0056] The fourth control devices 2-1 to 2-N control the process of receiving video captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the fourth snowbike drone SD4 via antenna AT4. Antenna AT4 transmits driving control signals issued from the driving command unit 303-6 to the fourth snowbike drone SD4. The video output unit 303-7 displays the video data captured by the FPV camera CAM transmitted from the transmitting / receiving antenna AT of the fourth snowbike drone SD4 on the monitor screen of the touch panel display 311A ​​or the 3D goggles 311B.

[0057] Figure 4 is a block diagram illustrating the configuration of the first to fourth unmanned snowbike drones SD1 to SD4, which are remotely controlled by the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control device 5-1 to 5-N, and the fourth control device 2-1 to 2-N shown in Figure 1.

[0058] In Figure 4, SD1 to SD4 are snowbike drones, each comprising a communication unit DR1, an FPV camera unit DR2, a GPS unit DR3, a driving control unit DR4, a main unit control unit DR5 that comprehensively controls these, an imaging memory unit DR6, a power supply unit DR7, a reception unit DR8, an unmanned snowbike drone control data storage unit DR9, a lighting control unit DR10, a first storage unit DR11, a second storage unit DR12, a third storage unit DR13, a fourth storage unit DR14, and an audio processing unit DR15. When the communication unit DR1 receives a driving instruction from the center server 1 via the network 21, it reads the search route information stored in advance in the driving control unit DR4, thereby controlling the engine start and engine stop of the snowbike drone body.

[0059] The main control unit DR5 controls the start of shooting by the FPV camera unit DR2, which has a 360-degree adjustable imaging direction, and the transmission of aerial data to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N.

[0060] Furthermore, the driving control unit DR4 has a function to acquire location information received by the GPS unit DR3 and determine whether or not the vehicle is traveling along a pre-stored search route.

[0061] Furthermore, the reception unit DR8 executes the process of receiving one of the exploration courses stored in the unmanned snowbike drone control data storage unit DR9 from the center server 1.

[0062] The imaging memory unit DR6 temporarily stores the imaging data captured by the FPV camera unit DR2 from the air, and, while synchronizing with the communication unit DR1, transmits the imaging data, which consists of a predetermined number of frames per second, to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N according to a 4G or 5G communication protocol.

[0063] The DR7 power unit primarily supplies power to the headlights, engine spark plugs, and heater, and also supplies power to other components as needed.

[0064] The AT is a transmitting and receiving antenna, and is configured to transmit captured video data (snowy road driving video data) to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N via an access point (not shown) using a wireless protocol, and to receive driving control information (standby instructions, route driving instructions, return instructions) from the center server 1.

[0065] However, the driving control instructions based on the registered snow-covered road search route may be configured to be stored in advance as a control program.

[0066] Furthermore, the driving control unit DR4 normally circles along a stored snow-covered road search route. However, through communication with the center server 1, if the center server 1 determines that the local weather conditions in the first to fourth search spot areas SP1 to SP4, specifically the amount of snow that can be driven and the wind speed forecast values ​​on the snow-covered road route obtained from the weather site, exceed a threshold, the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N are notified via the first alert to discontinue the snow-covered road search.

[0067] Therefore, when the main control unit DR5 receives the first alert from the center server 1, it notifies the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N of the reason for the search to be aborted.

[0068] The first snowbike drone SD1, the second snowbike drone SD2, the third snowbike drone SD3, and the fourth snowbike drone SD4 are equipped with a transmitting / receiving antenna AT that receives snow driving control signals transmitted from the corresponding first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control device 5-1~5-N, and the fourth control device 2-1~2-N, and also transmits video captured by the FPV camera CAM to the first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control device 5-1~5-N, and the fourth control device 2-1~2-N.

[0069] The driving control unit DR4 includes: a first control means that controls the driving direction by analyzing snow driving control information received from the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control device 5-1 to 5-N, and the fourth control device 2-1 to 2-N to control the direction of travel by controlling the rotation direction and amount of rotation of the rotation mechanism that rotates the direction control sled; a second control means that analyzes the snow driving control information to control the rotation start, rotation stop and rotation speed of the crawler for circulating drive of the track belt provided at the lower part of the rear of the frame at the rear of the main body of the snowbike; and a third control means that analyzes the snow driving control information to control the start and stop of the engine that drives the crawler.

[0070] Furthermore, the audio processing unit DR15 simultaneously performs audio processing, which includes a microphone M that collects the voice of a rescuer emitted from around the snowbike body, and a speaker SP that amplifies and outputs the audio transmitted from the first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control devices 5-1~5-N, and the fourth control devices 2-1~2-N; control that transmits the audio information collected by the microphone M via the transmitting / receiving antenna AT to the first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control devices 5-1~5-N, and the fourth control devices 2-1~2-N; and control that amplifies the audio information received from the first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control devices 5-1~5-N, and the fourth control devices 2-1~2-N via the transmitting / receiving antenna AT through the speaker SP.

[0071] Furthermore, the main body of the vehicle is equipped with a first storage compartment DR11 for storing predetermined rescue materials. Here, the predetermined rescue materials include a thermal blanket, a folding stretcher, a bone fixation device for fixing fractured bones, and a joint fixation device for fixing the cervical or lumbar vertebrae.

[0072] Furthermore, the main vehicle body may be configured to include a GPS transmitter that transmits location information for identifying the rescue location. The main vehicle body includes a floodlight, emergency radio communication equipment, a battery, and a second storage compartment DR12 for storing flares.

[0073] Furthermore, the main body of the vehicle is equipped with a third storage compartment DR13 for housing flares and fireplace materials. In addition, the main body of the vehicle is equipped with a fourth storage compartment DR14 for housing a winch DR16 for towing a foldable stretcher for transporting rescued persons.

[0074] Figure 5 is a block diagram illustrating the configuration of the center server 1 shown in Figure 1. In Figure 5, 11 is the communication unit, which controls the communication processing connected to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, which are connected to the network 21 as a communication medium.

[0075] 13 is the CPU, which starts the operating system (OS) stored in external memory 18 connected to the internal bus 12, and starts various applications installed via APIs. It also executes the processing of various programs deployed on RAM 16.

[0076] 14A is a point cloud database that stores image editing applications and procedures generated by the AI ​​support unit 19 on the exploration and photography video data captured by the FPV cameras CAM1 to CAM4 when the first to fourth snowbike drones SD1 to SD4 travel along the exploration and photography route via the network 21.

[0077] 14B is a search video database that temporarily stores search video data captured by FPV cameras CAM1 to CAM4 when traveling along a search route set adjacent to a snow search area transmitted via network 21 from one of the first to fourth snowbike drones SD1 to SD4.

[0078] Specifically, when one of the first to fourth snowbike drones SD1 to SD4 takes pictures along the exploration route selected by the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control device 5-1 to 5-N, and the fourth control device 2-1 to 2-N, it learns the captured video data and stores it in the exploration video database 14B, which functions as external memory.

[0079] 16 is expandable RAM, which stores the basic program of platform 1A installed in external memory 18.

[0080] Here, the RAM16 is configured with the AI ​​editing unit 16-1, the transmission unit 16-2, the first alert unit 16-3, the second alert unit 16-4, and the proposal unit 16-5. The CPU 13 executes these functions as appropriate according to the flowchart described later, thereby realizing various snow exploration video editing processes.

[0081] Specifically, AI editing unit 16-1 receives snow exploration video data from one of the first to fourth snowbike drones SD1 to SD4 and performs AI editing.

[0082] The first alert unit 16-3, when it determines that the weather conditions set for any of the first to fourth snowbike drones SD1 to SD4 have exceeded a threshold by comparing the drivable wind speed value with the wind speed forecast value along the flight route obtained from a weather site, executes a process to issue a first alert to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N to stop driving on snow.

[0083] Furthermore, the second alert unit 16-4 compares the search route traveled by the first to fourth snowbike drones SD1 to SD4 with the stored search route, and if it determines that the difference exceeds a threshold, it executes a process to issue a second alert to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, instructing them to correct the search route course.

[0084] The proposal unit 16-5 compares the flight route of any of the first to fourth snowbike drones SD1 to SD4 with the amount of snow in the search spot areas SP1 to SP4, and if it determines that the drone will be traveling on a snowy road with more than 2m of snow, it executes a process to propose a change of search route to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N.

[0085] 17 is a touch panel display that shows the program's startup status through various non-illustrated UI screens.

[0086] The following describes an example of the data collection process for point cloud measurement data stored in the point cloud database 14A shown in Figure 5. The process of collecting topographic information using point cloud measurements is carried out in the following steps.

[0087] Specifically, in the data acquisition process, point cloud data of the ground surface is collected using technologies such as laser scanning and LiDAR (Light Detection and Ranging). This stores the elevation and shape of the terrain in a point cloud database 14A, as shown in detail in Figure 5. Next, in the data processing process, the CPU 13 analyzes the collected point cloud data and extracts terrain features. At this stage, noise reduction and data interpolation are performed.

[0088] Next, in the terrain modeling process, CPU13 creates a 3D model of the terrain based on the processed data. This model visually represents the topography and features of the terrain.

[0089] Next, in the analysis and utilization process, CPU13 can use the created terrain model to analyze the terrain and, in conjunction with a Geographic Information System (GIS), determine a safe route for the first to fourth snowbike drones SD1 to SD4 to travel on snow, even at night.

[0090] Furthermore, by using multiple cameras mounted on the first to fourth snowbike drones SD1 to SD4, as well as the FPV camera CAM, to visually inspect the unevenness of the snow and perform AI image processing in conjunction with the AI ​​support unit 19, the image can be made more three-dimensional and easier to see on the display devices connected to the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N.

[0091] As a result, operators controlling the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, which are capable of executing functional programs such as those for the transmitter, can remotely control the snow-riding of the first to fourth snowbike drones SD1 to SD4 while viewing the FPV image on a monitor.

[0092] Furthermore, the goggle system, which can communicate with the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, may be configured so that the operator can view 3D video images instead of the above-mentioned monitor by wearing goggles that display the screen in front of their face.

[0093] Furthermore, by attaching altimeters to the first to fourth snowbike drones, SD1 to SD4, the following effects can also be expected.

[0094] The first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N can constantly monitor the altitude at which the first to fourth snowbike drones SD1 to SD4 are traveling by having the CPU 13 process information acquired from altimeters installed on the first to fourth snowbike drones SD1 to SD4 in real time. This allows them to notice steep slopes and obstacles, thus preventing accidents.

[0095] Furthermore, the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N accurately determine the snowmobile's position based on altitude information acquired from the altimeter, enabling more precise navigation.

[0096] Furthermore, by acquiring data from the altimeter, the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N can gain a more detailed understanding of the terrain's topography, enabling more efficient route selection.

[0097] Figures 6 to 8 are flowcharts illustrating the remote control processing method for the unmanned snowbike driving control system shown in this embodiment. This example corresponds to the remote control processing example on the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N shown in Figure 2. (1) to (20) indicate each step, and each step is realized by the CPU 301 executing a stored program.

[0098] First, the CPU 301 on the first control device 3-1~3-N, the second control device 4-1~4-N, the third control device 5-1~5-N, and the fourth control device 2-1~2-N perform pre-ride checks (including engine start, handle operation, brake operation, and headlight on / off confirmation) with the corresponding first to fourth snowbike drones SD1~SD4 (1).

[0099] Next, the CPU 301 on the first control unit 3-1~3-N, the second control unit 4-1~4-N, the third control unit 5-1~5-N, and the fourth control unit 2-1~2-N determines whether the communication response with the corresponding first to fourth snowbike drones SD1~SD4 is normal (2). If it determines that there is an abnormality in communication, this process is terminated in order to avoid danger due to driving abnormalities.

[0100] Meanwhile, in step (2), if the CPU 301 determines that there is no abnormality in communication, the CPU 301 on the side of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N set the driving route and rescue activity mode for the corresponding first to fourth snowbike drones SD1 to SD4 (3).

[0101] Next, the CPU 301 determines whether each operator has given a vertical movement instruction to the first to fourth snowbike drones SD1 to SD4 (4). If it determines that a vertical movement instruction has been given, the CPU 301 determines whether each operator has given a lateral movement instruction to the first to fourth snowbike drones SD1 to SD4 (5). If it determines that a lateral movement instruction has been given, the CPU 301 transmits snow driving control information to the corresponding first to fourth snowbike drones SD1 to SD4 (6).

[0102] Next, the CPU 301 determines whether it has received the driving video captured by each FPV camera CAM mounted on the first to fourth snowbike drones SD1 to SD4 (7). If the CPU 301 determines that it has received the driving video captured by each FPV camera CAM, it returns to step (4) and repeats the process.

[0103] On the other hand, if in step (4) the CPU 301 determines that no vertical movement instruction has been given, the CPU 301 determines whether the operator has given a stop instruction (8). If the CPU 301 determines that it has given a stop instruction, it proceeds to step (10) and determines the stopping position based on the GPS signals received at the stopping positions of the first to fourth snowbike drones SD1 to SD4. Next, the CPU 301 reports the GPS information corresponding to the stopping position to the center server 1 (11).

[0104] At this point, the center server 1 issues an instruction for the first to fourth snowbike drones SD1 to SD4, which are currently in motion, to gather at the aforementioned stopping position.

[0105] Next, once CPU301 confirms that all snowbike drones SD1 to SD4 have gathered at their stopping positions (12), CPU301 instructs the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N to each be assigned a role in the joint rescue operation (13).

[0106] Next, when the microphone M and speaker SP equipped in the first control device 3-1~3-N, the second control device 4-1~4-N, the third control device 5-1~5-N, and the fourth control device 2-1~2-N are activated and voices are made to the surrounding area, it is determined whether or not there is a response from a rescuer waiting for rescue by detecting voices requesting rescue from climbers, etc., who are waiting for rescue (14).

[0107] At this point, if CPU 301 determines that there is a rescuer response, it proceeds to step (17).

[0108] On the other hand, in step (14), if the CPU 301 determines that there is no response from the rescuer, the CPU 301 issues an instruction from the operator to change the search area (15) and notifies the center server 1 of the change in the search area (16).

[0109] On the other hand, if CPU301 determines that there is a response from the rescuers, in step (17), it uses the FPV camera CAM mounted on one of the first to fourth snowbike drones SD1 to SD4 to confirm the arrival of the rescuers via video. After reviewing the video from one of the FPV camera CAMs mounted on the first to fourth snowbike drones SD1 to SD4, if CPU301 determines that the stranded or injured person has been rescued (18), CPU301 instructs the team to return to the base camp at the foot of the mountain (not shown) (19).

[0110] Then, once it is confirmed by any of the FPV cameras (CAM) that all of the first to fourth snowbike drones, SD1 to SD4, have returned to base camp (20), this process is terminated.

[0111] Furthermore, it is envisioned that the injured persons who are rescued will receive care using medical equipment sets equipped by the first to fourth snowbike drones SD1 to SD4, as well as care to protect them from the cold, and then be towed by the first to fourth snowbike drones SD1 to SD4 while secured to a deployed stretcher.

[0112] Furthermore, when returning to base camp, team members can switch the driving modes of the 1st to 4th snowbike drones (SD1 to SD4) and drive back on the snow themselves.

[0113] On the other hand, if in step (8) the CPU 301 determines that it has not received a command to stop driving, then the CPU 301 determines whether to continue the search with the first to fourth snowbike drones SD1 to SD4 (9). If the CPU 301 determines to continue the search, it returns to step (4). If the CPU 301 determines not to continue the search, it proceeds to step (10).

[0114] As mentioned above, it is also possible to smoothly control the driving status of the first to fourth snowbike drones SD1 to SD4 in conjunction with the regular drones D1 to D4.

[0115] Figures 9 and 10 are flowcharts illustrating the remote control processing method for the unmanned driving control system of a snowbike in this embodiment. In this example, in order to enable driving assistance for the first to fourth snowbike drones SD1 to SD4 by the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control device 5-1 to 5-N, and the fourth control device 2-1 to 2-N shown in Figure 2, the system corresponds to a driving area registration process in which the first to fourth snowbike drones SD1 to SD4 are driven in a pre-set mountainous area to prompt avoidance of danger.

[0116] (21) to (39) indicate the respective steps, which are realized by the driving control unit DR4 executing the stored control program.

[0117] First, the operators of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N shown in Figure 2 repeatedly drive the first to fourth snowbike drones SD1 to SD4 continuously for a predetermined distance and then stop within a pre-set search area (assuming a search area determined from the location of the accident) during the summer months (21).

[0118] Next, the CPU 301 obtains information on the current stopping position and altitude from the central server 1 each time the first to fourth snowbike drones SD1 to SD4 stop (22). Then, the CPU 301 calculates and confirms the difference between the altitude information of the previous stopping position immediately before stopping and the altitude information of the current stopping position (23).

[0119] This allows us to determine the gradient in the direction that the first to fourth snowbike drones, SD1 to SD4, traveled.

[0120] Specifically, CPU 301 determines whether the difference is + or - (24), and if CPU 301 determines that the gradient is -, it determines that the direction of travel is downhill (downhill slope) (26), and proceeds to step (27).

[0121] On the other hand, in step (24), if the CPU 301 determines that the gradient is positive, it determines that the direction of travel is uphill (25). This gradient information is transmitted sequentially to the center server 1 as the first to fourth snowbike drones SD1 to SD4 travel. Even when encountering a situation during nighttime snow search where only a white world is visible on the touch panel displays 311A ​​of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N, the driving gradient icon described later can be displayed on the touch panel displays 311A, thereby preventing pilot errors due to whiteout by the operators of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N.

[0122] Next, the CPU 301 determines whether the direction in which the first to fourth snowbike drones SD1 to SD4 are traveling is towards a cliff based on the stored terrain information (27). If the CPU 301 determines that the direction in which the first to fourth snowbike drones SD1 to SD4 are traveling is not towards a cliff based on the stored terrain information, it proceeds to step (30).

[0123] Meanwhile, in step (27), if the CPU 301 determines that the direction in which the first to fourth snowbike drones SD1 to SD4 are traveling is towards a cliff based on the stored terrain information, it generates information to supplement the travel route (28).

[0124] Next, the CPU 301 determines whether the direction in which the first to fourth snowbike drones SD1 to SD4 are traveling is the direction of a fall (the area at risk of falling is determined by obtaining mountain accident information from a database) based on the stored terrain information (29). If the CPU 301 determines that the direction in which the first to fourth snowbike drones SD1 to SD4 are traveling is the direction of a fall, it returns to step (28).

[0125] On the other hand, in step (29), if the CPU 301 determines that the direction in which the first to fourth snowbike drones SD1 to SD4 are moving is not the direction of the fall, it proceeds to step (31) without generating information to supplement the travel route (30). Next, the CPU 301 determines whether the current position of the first to fourth snowbike drones SD1 to SD4 is at the northern edge of the search area (31). If the CPU 301 determines that the current position of the first to fourth snowbike drones SD1 to SD4 is at the northern edge of the search area, the CPU 301 confirms the northern side of the operation area (35) and proceeds to step (39).

[0126] On the other hand, in step (31), if CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are not at the northern end of the search area, CPU 301 determines whether the current positions of the first to fourth snowbike drones SD1 to SD4 are at the southern end of the search area (32). If CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are at the southern end of the search area, CPU 301 confirms the southern side of the search area (37) and proceeds to step (39).

[0127] On the other hand, in step (32), if CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are not at the southern end of the search area, CPU 301 determines whether the current positions of the first to fourth snowbike drones SD1 to SD4 are at the western end (33). If CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are at the western end of the search area, CPU 301 confirms the western side of the search area (36) and proceeds to step (39).

[0128] On the other hand, in step (33), if the CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are not at the western edge of the search area, the CPU 301 determines whether the current positions of the first to fourth snowbike drones SD1 to SD4 are at the eastern edge of the search area (34). If the CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are at the eastern edge of the search area, it determines the eastern side of the search area (38) and proceeds to step (39).

[0129] On the other hand, if the CPU 301 determines that the current positions of the first to fourth snowbike drones SD1 to SD4 are not at the eastern edge of the planned search area, it registers the safe driving area in which the first to fourth snowbike drones SD1 to SD4 can safely travel during nighttime searches of snowy mountains in the search video database 14B of the center server 1 (39), and then terminates this process.

[0130] This allows the operators of the first to fourth snowbike drones, SD1 to SD4, to avoid giving incorrect operating instructions even when a whiteout occurs on the snow and they cannot determine whether the direction of travel is towards a cliff or not.

[0131] Furthermore, by managing the safe driving areas stored in the point cloud database 14A so that they can be freely used by a wide range of stakeholders, it is possible to provide useful information for search and rescue operations in areas where mountain accidents frequently occur.

[0132] Figure 11 is a schematic diagram showing an example of driving guide icons displayed on the touch panel display 311A ​​and 3D goggles 311B of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N shown in Figure 2.

[0133] In Figure 11, the circle icons represent directions, with 0 being north, 90 east, 180 south, and 270 west. The arrows in the figure indicate gradients, while the other objects visually represent cliffs, downhill slopes, and uphill slopes in hilly areas.

[0134] This allows operators of the first control devices 3-1 to 3-N, the second control devices 4-1 to 4-N, the third control devices 5-1 to 5-N, and the fourth control devices 2-1 to 2-N to use the first to fourth snowbike drones SD1 to SD4, which move across snowy winter mountains, to visualize what kind of slope they will enter or whether they are approaching a cliff, even when viewing images in a whiteout, and to use this as auxiliary information to determine a safe route.

[0135] [Effects of the First Embodiment] According to this embodiment, a snowbike is equipped with an FPV camera (CAM) to create a drone, and a remote operator can control the power supply, direction, speed, stopping, and headlight operation of the snowbike using a control device. This allows for the transportation of necessary supplies to climbers and residents isolated by natural disasters at night in snowy regions during winter, as well as the transport of rescued individuals on stretchers, providing comprehensive, mobile, and rapid support for rescue operations in snow-covered disaster areas.

[0136] [Second Embodiment] In the above embodiment, the operator operating the first control devices 3-1~3-N, the second control devices 4-1~4-N, the third control device 5-1~5-N, and the fourth control device 2-1~2-N detailed an example of driving the first to fourth snowbike drones SD1~SD4 moving on the snow of a winter mountain. However, assuming a daytime search, the operator may prioritize the so-called normal search mode using drones D1~D4 to quickly determine the injured person's location, and then control the first to fourth snowbike drones SD1~SD4 moving on the snow of a winter mountain to expedite the search.

[0137] [Effects of the second embodiment] According to this embodiment, by using the first to fourth snowbike drones SD1 to SD4 to rescue injured persons before sunset and descending the mountain quickly, it is possible to shorten the search time while supporting search operations that have a high life-extending effect.

[0138] The disclosure relating to the present invention described above can be summarized to at least the following:

[0139] (1) An unmanned snowbike driving control system that allows a snowbike equipped with an FPV camera to drive on snow unmanned, by transmitting a snowbike driving control signal to remotely control the driving function of the snowbike, while viewing on a display connected to a control device that receives video data transmitted from the FPV camera via a predetermined communication medium, wherein the snowbike is equipped with a microphone that collects human voices generated around the body of the snowbike and a speaker that amplifies and outputs sound transmitted from the control device, and the sound processing means simultaneously performs control to transmit the sound information collected by the microphone to the control device via the transmitting and receiving antenna and control to amplify the sound information received from the control device via the transmitting and receiving antenna through the speaker.

[0140] (2) The snow bike is equipped with lighting means for illuminating the area ahead when traveling on snow at night, and the control device is characterized in that, when the snow bike is traveling at night, it sets a rescue activity mode to support rescue activities for people lost in the snow, controls the turning on or off of the lighting means, and controls the unmanned travel of the snow bike on snow along a search route identified by pre-stored terrain information while acquiring GPS signals.

[0141] (3) The control device is characterized in that, when the video data transmitted from the FPV camera is in a whiteout state, it stops the snow bike and simultaneously controls the control device to transmit the audio information collected by the microphone via the transmitting and receiving antenna, and controls the speaker to amplify the audio information received from the control device via the transmitting and receiving antenna.

[0142] (4) An unmanned driving control method for an unmanned snowbike driving system, which involves transmitting a snowbike equipped with an FPV camera to a snowbike via a predetermined communication medium, while viewing on a display connected to a control device that receives video data transmitted from the FPV camera, thereby transmitting a snowbike driving control signal to remotely control the driving function of the snowbike, wherein the snowbike includes a voice processing step that controls a microphone that collects human voices generated around the body of the snowbike and a speaker that amplifies and outputs sound transmitted from the control device, and the voice processing step is characterized by simultaneously performing control to transmit the voice information collected by the microphone via the transmitting and receiving antenna to the control device and control to amplify the voice information received from the control device via the transmitting and receiving antenna through the speaker. [Industrial applicability]

[0143] In the above embodiment, a system was described that uses multiple snowbike drones over a wide area to dynamically support search operations during nighttime snow searches. However, it is possible to similarly support search operations during the daytime as well. [Explanation of Symbols]

[0144] 1. Center Server 1A Platform 3-1~3-N First control device 4-1~4-N Second control device 5-1~5-N Third control device 2-1~2-N Fourth control device

Claims

1. An unmanned snowbike driving control system that allows a snowbike equipped with an FPV camera to drive on snow unmanned, by transmitting snow driving control signals to the snowbike to remotely control the driving functions of the snowbike, while viewing the data on a display connected to a control device that receives video data transmitted from the FPV camera via a predetermined communication medium, The aforementioned snow bike, The snowbike is equipped with a microphone that collects human voices generated around the main body of the snowbike, and a speaker that amplifies and outputs the sound transmitted from the control device, and a sound processing means that controls the speaker. The aforementioned audio processing means is characterized by simultaneously performing the following actions: controlling the microphone to transmit audio information collected by the microphone via the transmitting and receiving antenna to the control device, and controlling the speaker to amplify audio information received from the control device via the transmitting and receiving antenna.

2. The aforementioned snow bike, Equipped with lighting means to illuminate the road ahead when driving on snow at night, The control device is The snowbike unmanned driving control system according to claim 1, characterized in that, when the snowbike is driven at night, a rescue activity mode is set to support rescue activities for people lost in the snow, the lighting means is turned on or off, and the unmanned driving of the snowbike on the snow is controlled along a search route identified by pre-stored terrain information while acquiring GPS signals.

3. The unmanned snowbike driving control system according to claim 1 or 2, characterized in that the control device simultaneously performs the following actions when the video data transmitted from the FPV camera is in a whiteout state: stop the snowbike, transmit the audio information collected by the microphone via the transmitting and receiving antenna to the control device, and amplify the audio information received from the control device via the transmitting and receiving antenna through the speaker.

4. An unmanned driving control method for a snowbike, which involves transmitting a snowbike equipped with an FPV camera to a snowbike, while viewing the video data transmitted from the FPV camera on a display connected to the snowbike via a predetermined communication medium, thereby transmitting a snow driving control signal to the snowbike to remotely control the snowbike's driving functions, and enabling the snowbike to drive on snow unmanned, The aforementioned snow bike, The system includes a microphone that collects human voices generated around the body of the snowbike, and a speaker that amplifies and outputs the sound transmitted from the control device, and an audio processing step that controls these microphones. The method for controlling the unmanned operation of an unmanned snowbike operation system is characterized in that the voice processing step simultaneously performs the following actions: control to cause the microphone to transmit voice information collected via the transmitting and receiving antenna to the control device, and control to amplify the voice information received from the control device via the transmitting and receiving antenna through the speaker.

Images