Function expansion device for autonomous moving body, autonomous movement system, and autonomous movement method

Abstract

To provide a technology for expanding the functionality of autonomous movement without modifying a machine learning model of autonomous movement.SOLUTION: One of the function expansion devices for an autonomous moving body of the present invention is a function expansion device for the autonomous moving body that has an image acquisition unit, determines a moving route based on the acquired image, and performs autonomous movement, and includes: a communication unit that acquires current position information indicating the current position, destination information indicating the destination of the autonomous moving body, and image information indicating an image acquired by the image acquisition unit; a route search unit that calculates a movement route based on the destination information and current position information; and an image generation unit that generates a route image indicating the movement direction of the autonomous moving body based on the image information, and when a branching road is detected in the route image, the image generation unit generates a processed image by processing the route image, and the autonomous moving body moves based on the processed image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a function expansion device for an autonomous moving body, an autonomous moving system, and an autonomous moving method. [Background technology]

[0002] Conventionally, technologies related to moving objects that travel autonomously based on captured images have been put to practical use. In addition, technologies related to camera systems that support driver operation when multiple images of a road are captured, such as at an intersection, are also known.

[0003] As an example of the former technology, Patent Document 1 discloses an unmanned traveling cart. The unmanned traveling cart in Patent Document 1 is an unmanned traveling cart 1 that travels on a transport route within a travel area, and includes an imaging means 10 that captures images of guidelines and signs on the floor on both the left and right sides of the transport route, a range sensor 11 that detects obstacles ahead, a line identification means 12 that identifies the position of the guideline from the captured image, a sign identification means 13 that uses the image, a travel control means 14 that steers according to the position of the guideline and controls the steering direction and travel speed according to the type of sign, and an obstacle avoidance control means 15 that, if a detected obstacle is not on the destination, causes the cart to deviate from the transport route to avoid the obstacle and then returns to the original route, and, when stopping at the destination, causes the cart to stop until the obstacle is gone.

[0004] As an example of the latter technique, Patent Document 2 discloses an in-vehicle camera system. In the in-vehicle camera system of Patent Document 2, when a vehicle enters an intersection 52, an image range 53 in which a vehicle entering from the left direction can be confirmed among the omnidirectional images captured by the front camera 20A and an image cut-out range 54 in which a vehicle entering from the right direction can be confirmed are calculated. Then, an image 61 of the image cut-out range 53 and an image 62 of the image cut-out range 54 among the omnidirectional images captured by the front camera 20A are displayed on the display monitor in a dual-screen display. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-042165 [Patent Document 2] JP 2009-067292 A Summary of the Invention [Problem to be solved by the invention]

[0006] The technology of Patent Document 1 is an autonomous mobile system that uses an AI (machine learning) learning model that is based on the premise that the travel route is basically determined and that the system travels on roads without branching. Therefore, as limited by the premise conditions, it is difficult to apply the system to roads with branching roads. On the other hand, the technology in Patent Document 2 makes it possible to grasp the situation to the left and right of an intersection, but its purpose is to assist the driver in driving operations, and there is still room for improvement in terms of its use in autonomous driving.

[0007] Therefore, an object of the present invention is to provide a technology for expanding the function of autonomous movement without modifying the machine learning model of autonomous movement. [Means for solving the problem]

[0008] In order to solve the above problems, one representative function expansion device for an autonomous moving body of the present invention is a function expansion device for an autonomous moving body that has an image acquisition unit and determines a movement route based on an acquired image and moves autonomously, and is equipped with a communication unit that acquires destination information indicating the destination of the autonomous moving body, current location information indicating its current location, and image information indicating an image acquired by the image acquisition unit, a route search unit that calculates a movement route based on the destination information and current location information, and an image generation unit that generates a route image indicating the destination of the autonomous moving body based on the image information, and when a branching road is detected in the route image, the image generation unit generates a processed image by processing the route image, and the autonomous moving body moves based on the processed image. Effect of the Invention

[0009] According to the present invention, it is possible to expand the function of autonomous movement without modifying the machine learning model of autonomous movement. Problems, configurations and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an autonomous moving body. [Diagram 2] FIG. 2 is a flowchart showing the processing performed in the autonomous moving body. [Diagram 3] FIG. 3 is a diagram showing an example of an image acquired by the image acquisition unit. [Figure 4] FIG. 4 is a diagram showing an example of a configuration in which a function extension device is applied to an autonomous moving body. [Diagram 5] FIG. 5 is a diagram showing a flowchart of the processing performed in the autonomous mobile system. [Figure 6] FIG. 6 is a diagram showing a flowchart of an image generation process among flowcharts of the autonomous mobile system. [Figure 7] FIG. 7 is a diagram showing an example of a processed image generated by the image generating unit. [Figure 8] FIG. 8 is a diagram showing an example of a configuration in which a function extension device is applied to an autonomous moving body. [Figure 9] FIG. 9 is a functional block diagram showing an example of a configuration in which a function expansion device is applied to an autonomous moving body. [Figure 10] FIG. 10 is a flowchart showing the learning stage processing performed in the image analysis unit of the server. [Figure 11] FIG. 11 is a flowchart showing the image generation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiment. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals.

[0012] An autonomous mobile object is an object that selects its own route of travel without being operated by a user. For example, this includes self-driving automobiles, mobile robots that carry luggage in warehouses, and unmanned aerial vehicles such as drones.

[0013] (Conventional autonomous vehicle) A conventional autonomous moving body will be described with reference to Figures 1 to 3. Figure 1 is a diagram showing an example of the configuration of an autonomous moving body.

[0014] (composition) FIG. 1(a) shows a schematic diagram of an autonomous moving body, and FIG. 1(b) shows the autonomous moving body as a functional block. The autonomous moving body 1 includes an image acquisition unit 10, an image analysis unit 20, a drive control unit 30, and a drive unit 40. The image acquisition unit 10 is, for example, an imaging device. The image acquisition unit 10 is installed so as to face the moving direction of the autonomous moving body 1, and acquires an image including a travel path. Image information indicating the image acquired by the image acquisition unit 10 is sent to the image analysis unit 20. The image analysis unit 20 includes a trained model 21 and an inference unit 22. The image analysis unit 20 performs image analysis on the route image using a machine learning model, and extracts a travel path that is the route along which the autonomous moving body 1 should travel (move). A specific description will be given below.

[0015] The trained model 21 is, for example, a program indicating a trained machine learning model deployed in a memory, and performs a function by being executed by a processor. For example, a neural network can be used as the machine learning model. The neural network has an input layer, an intermediate layer, and an output layer, and an input value input to the input layer is weighted based on a plurality of variables in the intermediate layer, and an output value is derived from the output layer. In the present disclosure, a case where segmentation is used in image recognition will be described. In the stage of model training, parameters in the intermediate layer are adjusted so that a segmentation image in which a label (white line, driving lane, road shoulder, stop line, road sign, etc.) is attached to each pixel of the captured image is created.

[0016] The inference unit 22 uses the trained model to generate a segmentation image in which a label is attached to each pixel from the input image. In other words, at the stage of performing inference based on the machine learning model, a labeled image (segmentation image) is output for the input image.

[0017] The drive control unit 30 extracts a travel path through image analysis, and based on the results of the image analysis, controls the travel of the autonomous moving body 1. Specifically, the drive control unit 30 calculates parameters for controlling the travel of the autonomous moving body 1.

[0018] The driving unit 40 includes a power unit 41 that outputs power to move the autonomous moving body 1, and a direction control unit 42 that determines the moving direction of the autonomous moving body. The power unit 41 includes, for example, a rotating electric machine such as a motor and an internal combustion engine such as an engine. The driving control unit 30 controls the driving unit 40 so that the autonomous moving body 1 can achieve a predetermined moving speed. In addition, the direction control unit 42 includes a servo motor that controls the direction of the drive wheels. The driving control unit 30 controls the direction control unit 42 so that the autonomous moving body 1 moves in a direction along the travel path.

[0019] The drive unit 40 is not limited to the power unit 41 and the direction control unit 42, and may have other configurations. For example, it may include a braking control unit that controls a braking device.

[0020] (Autonomous movement processing) 2 is a diagram showing a flowchart of the processing performed in the autonomous moving body 1. The processing performed is mainly divided into processing before traveling (learning stage) and processing during traveling (inference and movement control).

[0021] As a process before traveling, learning data is created (step S1). The traveling path of the autonomous mobile body 1 may include straight lines, corners, intersections, and meandering roads. Data including the shapes of these expected traveling paths is acquired. In addition, images are acquired under various conditions, such as image acquisition conditions such as sensitivity and aperture, the state of the traveling path, and the environment in which the autonomous mobile body 1 exists.

[0022] Next, a machine learning model is created (step S2). Parameters are adjusted so that when the machine learning model is applied to the acquired image, a correct label is attached. The machine learning model with the adjusted parameters is used in the image analysis unit 20 as a trained model 21.

[0023] After the autonomous moving body 1 starts moving, processing during traveling is performed. Acquisition of images (step S3) is continuously performed by the image acquisition unit 10 while the autonomous moving body 1 is traveling. For example, it is possible to acquire images at predetermined intervals.

[0024] Next, image analysis is performed on the acquired image (step S4). The image analysis unit 20 extracts the travel path along which the autonomous moving body 1 travels from the acquired image.

[0025] Next, the drive control unit 30 determines a control signal to the drive unit 40 (step S5). Specifically, the drive control unit 30 calculates parameters for controlling the travel of the autonomous mobile body 1 for the travel path extracted through image analysis. The drive control unit 30 sends a control signal including the calculated parameters to the drive unit 40 to control the drive unit 40 (step 6).

[0026] (Image analysis) FIG. 3 is a diagram showing an example of an image acquired by the image acquisition unit 10. Although the image may include obstacles and signs in addition to the route, in this disclosure, only the route is displayed for simplicity. In the image, solid lines L1 and L2 and a dashed line L3 between the solid lines L1 and L2 are detected. The area between the solid lines L1 and L2 indicates the travel path of the autonomous mobile body 1. The autonomous mobile body 1 is set to travel within the range of the travel path. The dashed line L3 is set in the center part of the travel path. For example, by controlling the autonomous mobile body 1 so as not to deviate from the travel path based on the dashed line L3, the autonomous mobile body 1 can achieve stable travel within the travel path. By extracting the solid lines L1 and L2 and the dashed line L3 from the image in this way, the drive control unit 30 can move the autonomous mobile body 1 stably.

[0027] 3(a) shows the travel path extending to the front left side of the autonomous mobile body, and FIG. 3(b) shows the travel path extending to the front right side of the autonomous mobile body 1. The travel path of the autonomous mobile body 1 is basically one path that does not branch out.

[0028] In addition, although the case where the area between the solid lines L1 and L2 is a travel path has been described, the present disclosure is not limited to this. The area between the solid lines L1 and the dashed lines L3 may be a travel path for a moving body heading in one direction, and the area between the dashed lines L3 and the solid lines L2 may be a travel path for a moving body heading in the other direction. In addition, when an unmanned aerial vehicle is applied as an autonomous mobile body, the travel path refers to the space in which flight is permitted.

[0029] [First embodiment] (Functional expansion device and autonomous mobile system) The function expansion device according to the first embodiment will be described with reference to Figs. 4 to 7. Fig. 4 is a diagram showing an example of a configuration in which the function expansion device is applied to an autonomous moving body. The autonomous moving body 1a moves autonomously by utilizing a server 80 and a database 90. The function expansion device may refer to the server 80, or may refer to a device including the server 80 and the database 90. A system including the autonomous moving body 1a, the server 80, and the database 90 can also be called an autonomous moving system 100. The server 80 is a function expansion device for the autonomous moving body 1a that has an image acquisition unit 10 and determines a moving route based on an acquired image and moves autonomously. The server 80 includes a communication unit 81 that receives destination information indicating the destination of the autonomous moving body 1a, current position information indicating the current position, and image information indicating an image acquired by the image acquisition unit 10, a route search unit 82 that calculates a moving route based on the destination information and the current position information, and an image generation unit 83 that generates a route image indicating the moving destination of the autonomous moving body 1a based on the image information. When a branch road is detected in the route image, the image generating unit 83 generates a processed image by processing the route image, and the autonomous moving body 1a moves based on the processed image. Specific explanation will be given below.

[0030] (composition) FIG. 4(a) shows the configuration in a schematic manner, and FIG. 4(b) shows the configuration as a functional block. According to FIG. 4(b), the autonomous moving body 1a is different from the configuration of the conventional autonomous moving body 1 in that it has a GPS (positioning unit) 50, an operation unit 60, and a communication unit 70. The image acquisition unit 10, the image analysis unit 20, the drive control unit 30, and the drive unit 40 are common to the conventional autonomous moving body 1. The server 80 also has a communication unit 81, a route search unit 82, and an image generation unit 83. Although the database 90 is shown as a separate configuration from the server 80, the server 80 may have a function equivalent to the database. The configuration of the server 80 can be appropriately selected from a case where it is configured as hardware, a case where it is realized by a cloud service, and the like.

[0031] The GPS 50 measures the current position of the autonomous moving body 1a and acquires current position information. The GPS 50 transmits the current position information indicating the current position to the communication unit 70, and the communication unit 70 transmits the current position information to the server 80. Furthermore, the operation unit 60 is operated by a user of the autonomous moving body 1a and accepts input of destination information. The user sets the destination of the autonomous moving body 1a via the operation unit 60. The operation unit 60 sends destination information indicating the destination to the communication unit 70, and the communication unit 70 sends the destination information to the server 80. Note that the operation unit 60 may be installed on the autonomous moving body 1a, or may be set in a location different from the autonomous moving body 1a (for example, on the server 80). Also, the operation unit 60 may be set as an external control unit capable of communicating with the autonomous moving body 1a. The image acquisition unit 10 acquires an image with a viewing angle facing the traveling direction of the autonomous moving body 1a. The image acquisition unit 10 transmits image information indicating the image to the communication unit 70, and the communication unit 70 transmits the image information to the server 80.

[0032] The communication unit 81 of the server 80 acquires the destination information, current position information, and image information sent from the communication unit 70 of the autonomous mobile body 1a. In the following description, the communication unit 70 and the communication unit 81 will be described as performing wireless communication, but the present disclosure is not limited to this. A communication method other than wireless communication may be adopted. Communication standards such as WiMAX (registered trademark) and wireless LAN may be used as the wireless communication method. It is also possible to install the communication unit 81 of the server 80 on a road and perform road-to-vehicle communication between the server 80 and the autonomous mobile body 1a.

[0033] The route search unit 82 calculates a moving route of the autonomous moving body 1a based on at least the destination information and the current position information. The method of calculating the moving route is not particularly limited. For example, the route search unit can calculate the moving route based on the map information, the destination information, and the current position information. Specifically, the route search unit 82 can use the map information and traffic information stored in the database 90 to calculate a route in which indicators such as a moving time and a moving distance are within a specific range as a moving route that is a route along which the autonomous moving body 1a plans to move. In addition, the route search unit 82 can connect to a route search service provided by a content provider and calculate a route that satisfies a predetermined condition as a moving route of the autonomous moving body 1a.

[0034] The destination information is sent to the server 80 before the autonomous moving body 1a starts traveling, and the image information and the current position information are continuously sent to the server 80 at a predetermined interval while the autonomous moving body 1a is traveling. There are cases where the traveling is interrupted midway and the destination information is reset. In such a case, the route search unit 82 calculates a new travel route from the travel route set before the autonomous moving body 1a starts traveling, based on the current position information and the destination information.

[0035] The image generating unit 83 generates a route image, which is an image showing the destination of the autonomous moving body 1a, based on the image information. When a branching road is detected in the route image, the image generating unit 83 generates a processed image by processing the route image. The processing in the image generating unit 83 will be described later. The route image and the processed image are sent to the autonomous moving body 1a via the communication unit 81.

[0036] The image generating unit 83 mainly has two functions: extraction of the road and generation of the image. An edge detection method is used for extraction of the road. Methods such as filtering and brightness analysis, and a method using a machine learning model like the image analyzing unit 20 can also be adopted.

[0037] The autonomous moving body 1a moves based on the processed image. Specifically, the route image and the processed image are sent to the image analysis unit 20 via the communication unit 70. The image analysis unit 20 performs image analysis on the route image and the processed image, and extracts a travel path through the image analysis. After the travel path is extracted, processing is performed in the drive control unit 30 and the drive unit 40.

[0038] (Autonomous movement processing) Fig. 5 is a diagram showing a flowchart of the processing performed in the autonomous mobile system 100. Fig. 6 is a diagram showing a flowchart of the image generation processing among the flowcharts of the autonomous mobile system 100.

[0039] Steps S1 to S6 in Fig. 5 correspond to steps S1 to S6 in Fig. 2, respectively. The autonomous moving body 1a performs pre-travel processing (learning stage) and traveling processing (inference and movement control) similar to the conventional autonomous moving body 1. However, it differs from the conventional autonomous moving body 1 in that the image acquired in image acquisition (step S3) is not directly subjected to image analysis (step S4) but an image processed by the server 80 may be used.

[0040] The destination setting (step S7) is a process of setting the destination of the autonomous moving body 1a. The destination is set by a user's operation performed on the operation unit 60. The destination information is sent from the communication unit 70 to the communication unit 81 of the server 80.

[0041] The route search (step S8) is a process of calculating a moving route based on destination information. The route search unit 82 calculates a moving route of the autonomous moving body 1a based on the destination information and the current position information.

[0042] The image generation process (step S9) is a process of image analysis and image generation in the image generation unit 83. Fig. 6 shows a flowchart of the image generation process.

[0043] The image generating unit 83 acquires image information (step S3) and current location information (step S10). The image generating unit 83 uses the current location information to determine whether the destination has been reached (step S11). The image generating unit 83 refers to the map information stored in the database 90 and the destination information acquired in the destination setting (step S7), and compares it with the current location information to determine whether the destination has been reached (step S11).

[0044] If it is determined that the destination has not been reached (No in step S11), the image generating unit 83 determines whether or not a branch road appears on the actual route based on the current location information (step S12). For example, it is possible to adopt a criterion for the determination, such as whether or not a branch road appears within a range of a predetermined distance from the current location, or whether or not a branch road exists within a range several times the interval at which images are acquired.

[0045] The image generating unit 83 calculates a branch road from the route image (step S13). If a branch road is included in the route image (Yes in step S14), the image generating unit 83 performs image processing to process the image into one including only one road in the traveling direction of the autonomous moving body 1a (step S15). Specifically, the image generating unit 83 processes the image to erase a route (traveling route) that is not included in the travel route among at least two routes (traveling routes) included in the branch road, thereby generating a processed image.

[0046] If it is determined that no branch road is included (No in step S12 and No in step S14), the image generating unit 83 does not process the acquired image and outputs it as a route image without processing it.

[0047] The processed image and the route image are sent to the image analysis unit 20 of the autonomous moving body 1a. The autonomous moving body 1a moves based on the processed image or the route image.

[0048] When it is determined that the autonomous moving body 1a has arrived at the destination (Yes in step S11), it is necessary to stop the traveling of the autonomous moving body 1a. In such a case, it is also possible to control the image generating unit 83 to generate a processed image including a stop pattern (for example, a stop line or an area where parking is possible) on the traveling road, and to control the image analyzing unit 20 to stop the output of the power unit 41. When it is not assumed that the stopping pattern will be identified in the machine learning model, it is also possible to generate a processed image showing a curve, and control the autonomous moving body 1a to go around the destination.

[0049] (Edited image) FIG. 7 is a diagram showing an example of a processed image generated by the image generating unit 83. FIG. 7(a) shows an image acquired by the image acquiring unit 10, and FIG. 7(b) and FIG. 7(c) show an example of processing by the image generating unit 83. In FIG. 7(a), a branching road that branches into two routes from the front to the back (from the bottom to the top of the drawing) is shown. FIG. 7(b) shows the image of FIG. 7(a) processed, with the route extending to the right being erased. This is because the moving route is the left route, and the right route is not a moving route, so the right route is erased. On the other hand, FIG. 7(c) shows the image of FIG. 7(a) processed, with the route extending to the left being erased. This is because the moving route is the right route, and the left route is not a moving route, so the left route is erased.

[0050] (Action and effect) According to the present disclosure, it is possible to perform desired autonomous movement even when a branching road occurs in an image, while utilizing a machine learning model used in a conventional autonomous moving body. In addition, the image acquisition function and drive control function of the conventional autonomous moving body can basically be utilized as is. If the present disclosure is applied to autonomous driving, the autonomous moving body 1 corresponds to a vehicle equivalent to level 1 (the system assists in acceleration / deceleration or steering) or level 2 (the system assists in acceleration / deceleration and steering). The autonomous moving body 1a in the autonomous moving system 100 corresponds to a vehicle equivalent to level 3 (autonomous driving under specific conditions). In this way, according to the present invention, it is possible to expand the function of autonomous movement without modifying the learning model of the autonomous movement.

[0051] In addition, when a travel route needs to be changed when using a conventional autonomous moving body, infrastructure preparation work such as resetting indicators (solid and dashed lines in this disclosure) on the route is required so that the travel route can be estimated by a machine learning model. According to the present disclosure, it is possible to respond to changes in the travel route by placing an infinite number of indicators on the route to identify the travel route. Since the autonomous moving body is not limited to one travel route and can move autonomously along the calculated travel route, it is possible to set a travel route with a high degree of freedom.

[0052] [Second embodiment] The second embodiment differs from the first embodiment in that the functions of the image analysis unit and the drive control unit are provided in a device other than the autonomous moving body. In the following description, the same or equivalent components as those in the first embodiment described above are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0053] (composition) An autonomous mobile system 100a according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a configuration in which a function expansion device is applied to an autonomous mobile body. Fig. 8(a) shows the configuration in a schematic manner, and Fig. 8(b) shows the configuration as a functional block.

[0054] The autonomous mobile body 1b has an image acquisition unit 10, a drive unit 40, a GPS 50, an operation unit 60, and a communication unit 70. While the autonomous mobile body 1a of the first embodiment includes an image analysis unit 20 and a drive control unit 30, the autonomous mobile body 1b of the second embodiment does not include the image analysis unit 20 and the drive control unit 30. The image analysis unit 20 and the drive control unit 30 constitute an autonomous mobile system 100a as an image analysis device 2 that is a device separate from the autonomous mobile body 1b.

[0055] (Autonomous movement processing) As a processing method, the route image or processed image generated in the server 80 is sent to the image analysis device 2. In the image analysis device 2, the image analysis unit 20 extracts the travel path of the autonomous moving body 1 from the acquired image. Then, the drive control unit 30 determines a control signal to the drive unit 40. Specifically, the drive control unit 30 calculates parameters for controlling the travel of the autonomous moving body 1 for the travel path extracted through image analysis. The control signal including the calculated parameters is sent from the communication unit 23 to the communication unit 70 of the autonomous moving body 1b. The drive unit 40 of the autonomous moving body 1b drives the power unit 41 and the direction control unit 42 based on the control signal. Note that the communication unit 23 can apply an appropriately selected communication method, similar to the communication units 70 and 81.

[0056] (Action and effect) The processes performed by the image analysis unit 20 and the drive control unit 30 are relatively heavy-duty processes. When hardware such as a processor and a memory for performing these processes is installed in an autonomous moving body, the specifications of the hardware may be limited. In contrast, in the second embodiment, the image analysis unit 20 and the drive control unit 30 are configured as independent devices separate from the autonomous moving body. Since there are no restrictions on the hardware configuration, it is possible to avoid restrictions on specifications. Therefore, for example, it is possible to realize an autonomous moving system with improved real-time performance.

[0057] [Third embodiment] The third embodiment differs from the first embodiment in the way in which a processed image is generated on the server. In the following description, the same or equivalent components as those in the first embodiment described above are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0058] (composition) The configuration of an autonomous mobile system 100b according to the third embodiment will be described with reference to Fig. 9. Fig. 9 is a functional block diagram showing an example of a configuration in which a function expansion device is applied to an autonomous mobile body. An image analysis unit 83a of a server 80a includes a determination unit 84, a trained model 85, and an inference unit 86.

[0059] (Image generation processing) The image generation process according to the third embodiment will be described with reference to Figs. 10 and 11. Fig. 10 is a diagram showing a flowchart of the learning stage process performed in the image analysis unit 83a of the server 80a. Fig. 10 shows the process before traveling, and can be performed in parallel with, for example, steps S1 and S2, and steps S7 and S8 in Fig. 5.

[0060] First, teacher data is acquired (step S31). The teacher data is image data including a route and objects that may be captured along the route. For example, when assuming a vehicle as the autonomous moving body 1a, image data including roads, telephone poles, traffic lights, cars, people, the sky, buildings, etc. is acquired.

[0061] Next, a machine learning model is created (step S32). As the machine learning model, it is possible to apply machine learning models such as a neural network, a decision tree, and a support vector machine. In the second embodiment, object detection is mainly performed in the image analysis. It is possible to select or combine appropriate machine learning models for use. The trained model 85 in FIG. 9 is a trained machine learning model whose parameters have been adjusted through the learning stages of steps S31 and S32, and is, for example, a program indicating the trained machine learning model deployed in a memory.

[0062] Fig. 11 is a diagram showing a flowchart of the image generation process. The flowchart in Fig. 11 shows the process performed in the image analysis unit 83a, and in particular, steps S91 to S93 are processes performed in place of the image processing process (step S5 in Fig. 5). In Fig. 11, in addition to an explanation of the process, changes in the image during the process are also shown.

[0063] Image P1 is an example of an image captured when image capture (step S3) is performed. Image P1 includes a road b1, a roadside b2, a building b3, and a sky b4.

[0064] The determination unit 84 analyzes the image P1 and detects an object (step S91). A trained model 85 is used for object detection. The determination unit 84 detects a road b1, a roadside b2, a building b3, and a sky b4. The determination unit 84 also detects the shape of the detected object, and for example, for the road b1, it detects that there is a fork in the road that branches into two as it goes from the front to the back, as shown in the image P2.

[0065] Next, the determination unit 84 deletes unnecessary parts included in the image (step S92). The determination unit 84 acquires information such as the movement route and current position information of the autonomous moving body 1a from the route search unit 82. In order to generate a route image, the determination unit 84 deletes routes that are not included in the movement route from the road b1. The determination unit 84 deletes the road b1a going to the left. As shown in the deleted image P3, the deleted parts are left blank.

[0066] Next, the determination unit 84 fills in the blank areas (step S93). The determination unit 84 fills in the blank areas to make the image more natural. Specifically, the learned model 85 is used to estimate what color the blank areas should be filled in based on the objects around the blank image, and the blank areas are filled in based on the estimation result. As shown in image P4, the road b1a is filled in with the same type of image as the road shoulder b2.

[0067] The image P4 is sent to the image analysis unit 20 of the autonomous moving body 1 as a processed image.

[0068] (Action and effect) Depending on the number of samples of data to be used as training data and the type of machine learning model selected, it may be difficult to ensure sufficient accuracy during estimation. In the present disclosure, the image processing method is not limited to segmentation in the first embodiment, and object detection as in the third embodiment can also be adopted. Since the object detection method can be selected according to the environment and conditions in which the autonomous mobile system is used, it is possible to improve the independence and reusability of the server as a function expansion device.

[0069] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention.

[0070] The following are some possible embodiments of the present invention, but the present invention is not limited to these. (Aspect 1) A function expansion device for an autonomous moving body that has an image acquisition unit, determines a moving route based on an acquired image, and moves autonomously, a communication unit that acquires destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; A route search unit that calculates a travel route based on the destination information and the current position information; an image generating unit that generates a route image indicating a destination of the autonomous moving body based on the image information, The image generating unit includes: When a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous moving body moves based on the processed image. (Aspect 2) The function expansion device for an autonomous moving body according to aspect 1, wherein the image generation unit generates the processed image by processing the image so as to erase a route that is not included in the travel route out of at least two routes included in the branch route. (Aspect 3) A database in which map information is stored, 3. The function expansion device for an autonomous moving body according to aspect 1 or 2, wherein the route search unit calculates the moving route based on the map information, the destination information, and the current position information. (Aspect 4) The autonomous moving body is an operation unit that accepts an input of the destination information; a positioning unit for acquiring the current location information; an image analysis unit that performs image analysis on the route image using a machine learning model and extracts a travel path that is a route to be traveled; A function expansion device according to any one of aspects 1 to 3, further comprising: a drive control unit that controls travel based on a result of the image analysis. (Aspect 5) An autonomous mobility system, comprising: an autonomous moving body having an image acquisition unit, which determines a moving route based on the acquired image and moves autonomously; a communication unit that acquires destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; A route search unit that calculates a travel route based on the destination information and the current position information; an image generating unit that generates a route image indicating a destination of the autonomous moving body based on the image information, The image generating unit includes: If a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous mobile body moves based on the processed image. (Aspect 6) An autonomous movement method for an autonomous moving body having an image acquisition unit, determining a movement path based on an acquired image and performing autonomous movement, comprising: a communication step of acquiring destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; a route search step of calculating a travel route based on the destination information and the current position information; and an image generating step of generating a route image showing a destination of the autonomous moving body based on the image information, In the image generating step, If a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous moving body moves based on the processed image. [Explanation of symbols]

[0071] 1, 1a, 1b: autonomous moving body, 2: image analysis device, 10: image acquisition unit, 20 drive control unit 20: image analysis unit, 21: trained model, 22: inference unit, 23: communication unit, 30: drive control unit, 40: drive unit, 41: power unit, 42: direction control unit, 60: operation unit, 70: communication unit, 80, 80a: server, 81: communication unit, 82: route search unit, 83, 83a: image analysis unit, 84: judgment unit, 85: trained model, 86: inference unit, 90: database, 100, 100a, 100b: autonomous mobile system

Claims

1. A function expansion device for an autonomous moving body that has an image acquisition unit, determines a moving route based on an acquired image, and moves autonomously, a communication unit that acquires destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; A route search unit that calculates a travel route based on the destination information and the current position information; an image generating unit that generates a route image indicating a destination of the autonomous moving body based on the image information, The image generating unit includes: When a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous moving body moves based on the processed image.

2. The function expansion device for an autonomous moving body according to claim 1 , wherein the image generation unit generates the processed image by processing the at least two routes included in the branch path so as to erase a route that is not included in the travel route.

3. A database in which map information is stored, The function expansion device for an autonomous moving body according to claim 1 , wherein the route search unit calculates the travel route based on the map information, the destination information, and the current position information.

4. The autonomous moving body is an operation unit that accepts an input of the destination information; a positioning unit for acquiring the current location information; an image analysis unit that performs image analysis on the route image using a machine learning model and extracts a travel path that is a route to be traveled; The function expansion device according to claim 1 , further comprising: a drive control unit that controls travel based on a result of the image analysis.

5. An autonomous mobility system, comprising: an autonomous moving body having an image acquisition unit, which determines a moving route based on the acquired image and moves autonomously; a communication unit that acquires destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; A route search unit that calculates a travel route based on the destination information and the current position information; an image generating unit that generates a route image indicating a destination of the autonomous moving body based on the image information, The image generating unit includes: If a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous mobile body moves based on the processed image.

6. An autonomous movement method for an autonomous moving body having an image acquisition unit, determining a movement path based on an acquired image and performing autonomous movement, comprising: a communication step of acquiring destination information indicating a destination of the autonomous moving body, current location information indicating a current location of the autonomous moving body, and image information indicating an image acquired by an image acquisition unit; a route search step of calculating a travel route based on the destination information and the current position information; and an image generating step of generating a route image showing a destination of the autonomous moving body based on the image information, In the image generating step, If a branch road is detected in the route image, a processed image is generated by processing the route image; The autonomous moving body moves based on the processed image.

Images