Image generation device, image generation method, image generation program and navigation device

Abstract

To provide an image generation device that creates an image that can be used for navigation based on an image captured from a vehicle.SOLUTION: An image generation device comprises: a storage section which stores an image captured from a vehicle, a condition for generating an illustration image based on the captured image, and a learning model learning a relation between the captured image and the illustration image based on the condition; an acquisition section which acquires a basic image captured from the vehicle; a reception section which receives input of a generation condition for generating the illustration image; and a generation section which generates a generated illustration image based on the basic image meeting the generation condition by inputting the basic image and the generation condition into the learning model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image generating device that generates an illustration image from a captured image, an image generating method, and an image generating program. [Background technology]

[0002] Conventionally, images for guidance in navigation devices have been created manually or by requesting someone with the skills to create them. However, this creation process is time-consuming and cumbersome. Therefore, it is considered to use captured images. Patent Documents 1 and 2 disclose technologies for generating illustrated images from captured images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-157705 [Patent Document 2] JP 2003-203211 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, simply converting a captured image into an illustrated image as in Patent Documents 1 and 2 above poses a problem in that the image is unusable for navigation purposes.

[0005] Therefore, the present invention has been made in consideration of the above problems, and aims to provide an image generation device, an image generation method, and an image generation program that can generate an image that can be used for navigation, etc. from a captured image. [Means for solving the problem]

[0006] In order to solve the above problem, an image generation device according to one embodiment of the present invention includes a memory unit that stores a learning model that learns the relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions, an acquisition unit that acquires a basic image captured from the vehicle, a reception unit that accepts input of generation conditions for generating the illustrative image, and a generation unit that inputs the basic image and the generation conditions into the learning model and generates a generated illustrative image based on the basic image in accordance with the generation conditions.

[0007] In addition, in order to solve the above problem, an image generation method according to one embodiment of the present invention executes the following steps: a storage step in which a computer stores a learning model that has learned the relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions; an acquisition step in which a basic image captured from the vehicle is acquired; a reception step in which an input of generation conditions for generating an illustrative image is accepted; and a generation step in which the basic image and the generation conditions are input into the learning model to generate a generated illustrative image based on the basic image in accordance with the generation conditions.

[0008] In addition, in order to solve the above problem, an image generation program according to one embodiment of the present invention provides a computer with a memory function for storing a learning model that has learned the relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions, an acquisition function for acquiring a basic image captured from the vehicle, a reception function for receiving input of generation conditions for generating an illustrative image, and a generation function for inputting the basic image and the generation conditions into the learning model and generating a generated illustrative image based on the basic image in accordance with the generation conditions.

[0009] In the image generating device, the generating section may generate a generated illustrated image based on the captured image so as to clarify a road.

[0010] In the image generating device, the generating unit may generate the generated illustrative image so as to clarify road markings on the road.

[0011] In the image generating device, the generating section may generate the generated illustrative image so as to simplify features other than roads.

[0012] In the image generating device, the generating section may generate the generated illustrated image by deleting a predetermined object included in the captured image based on the captured image.

[0013] The image generating device may further include an output section that outputs the generated illustrative image.

[0014] In addition, a navigation device according to one embodiment of the present invention includes a memory unit that stores registered map information based on the location at which an image used to generate a generated illustrative image was captured by the image generation device, an acquisition unit that acquires location information, and an output unit that outputs a generated illustrative image corresponding to the location when the location information is a location that satisfies a predetermined condition. Effect of the Invention

[0015] An image generating device according to one aspect of the present invention can generate an illustrative image of a desired aspect from an image captured by an imaging device of a vehicle while the vehicle is traveling, by inputting conditions for the image to be generated. In other words, the image generating device can easily generate an illustrative image that can be used in a navigation device or the like, and that would require a lot of effort if created manually. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image generating device. [Diagram 2] 13 is a flowchart showing an operation example of the image generating apparatus, which is an operation example of a learning process. [Diagram 3] 10 is a flowchart showing an operation example of the image generating apparatus, that is, an operation example of an image generating process. [Figure 4] 4(a) is a diagram showing an example of a captured image, and FIG. 4(b) is a diagram showing an example of an illustration image generated based on the captured image of FIG. [Diagram 5] 5(a) is a diagram showing an example of a captured image, and (b) is a diagram showing an example of an illustration image generated based on the captured image of FIG. [Figure 6] FIG. 13 is a block diagram showing another example of the configuration of the image generating device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an image generating device according to an aspect of the present invention will be described in detail with reference to the drawings.

[0018] <Embodiment> <Configuration> An image generating device according to an embodiment of the present invention includes a storage unit (see 104 in FIG. 1) that stores a learning model (see 141 in FIG. 1) that learns the relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions, an acquisition unit (see 105 in FIG. 1) that acquires a basic image captured from the vehicle, a reception unit that receives input of the generation conditions for generating the illustrative image, and a generation unit (see 105 in FIG. 1) that inputs the basic image and the generation conditions to the learning model and generates a generated illustrative image based on the basic image according to the generation conditions. The details will be described below.

[0019] 1 is a block diagram showing an example of the configuration of an image generating device 100. As shown in FIG. 1, the image generating device 100 includes a communication unit 101, an input unit 102, an output unit 103, a storage unit 104, and a CPU 105.

[0020] The image generating device 100 is a computer system realized by a server device, a PC, etc., but is not limited thereto, and may be realized by a mobile terminal such as a smartphone, a tablet terminal, etc. As an example, the image generating device 100 is a device that generates an illustrated image that can be used as a CG image for guidance in a navigation device, etc., based on an image captured from a vehicle, i.e., an image captured by a camera (imaging unit) mounted on the vehicle, etc. Note that the captured image only needs to include at least a road, and is not limited to an image mounted on a vehicle, etc., and may be an image captured by a person holding a camera, or may be an image captured by a surveillance camera installed on a building, etc.

[0021] The communication unit 101 has a function of transmitting and receiving information to and from other devices through wired or wireless communication. For example, the communication unit 101 may receive a captured image from an external device (for example, a traveling vehicle, a vehicle camera, or a device that stores captured images including a road, etc.) and transmit the captured image to the CPU 105. For example, the communication unit 101 may receive condition information indicating conditions for generating an illustrative image from the external device and transmit the condition information to the CPU 105. For example, the communication unit 101 may transmit specified information to a specified destination according to an instruction from the CPU 105. For example, the communication unit 101 may transmit a generated illustrative image.

[0022] The input unit 102 has a function of accepting input from a user of the image generating device 100 and transmitting the input to the CPU 105. The input unit 102 can be realized, for example, by hardware keys provided in the image generating device 100, or soft keys such as a touch panel or touch keys. Note that the input to the input unit 102 may be voice input, in which case the input unit 102 is realized by a microphone. The input unit 102 may accept, for example, input of conditions for generating an illustration image from a captured image and transmit the input to the CPU 105.

[0023] The output unit 103 has a function of outputting instructed data according to an instruction from the CPU 105. The output unit 103 may output, for example, an illustrated image generated by the CPU 105. The output of information by the output unit 103 may be output in the form of characters or images to a monitor (display device) or the like that is attached to or connected to the image generating device 100, may be output of audio from a speaker that is attached to or connected to the image generating device 100, or may be output of information by communication to an external device via the communication unit 101.

[0024] The storage unit 104 is a recording medium that stores various programs and various data required for the operation of the image generating device 100. The storage unit 104 is realized by, for example, a hard disc drive (HDD), a solid state drive (SSD), a flash memory, etc., but is not limited to these.

[0025] The memory unit 104 may store a learning model 141.

[0026] The learning model 141 may be a model that has learned the correspondence between at least a captured image of the outside of the vehicle taken from the vehicle and an illustrated image generated based on the captured image. That is, the learning model 141 may be a learning model that has learned a combination of a captured image and an illustrated image to be generated based on the captured image as training data. The illustrated image used as the training data may be a CG created by a human hand by looking at the captured image. By learning the training data, the learning model 141 becomes a model that can receive an input of a captured image and generate (or convert) an illustrated image. In this case, the illustrated image is basically an illustration from the same angle of view as the captured image. The illustrated image used as the training data may be an image that a navigation device or the like presents to a driver or the like when providing guidance. Therefore, it is preferable that the illustrated image is an image that is useful for the driver, and it is preferable that the information necessary for driving is made clear and the information unnecessary for driving is made unclear. As an example, the clarity of the information necessary for driving may be clarification of the outline of the road, clarification of the dividing line, and clarification of the road sign. The clarification of road contours and demarcation lines may be, for example, the clarification of boundary lines, and may be the thickening of line segments or the darkening of color to make them easier to see. The clarification of road signs may be the sharpening of character strings written on signboards as road signs, or the sharpening of blurred content in captured images. In contrast, the blurring of information unnecessary for driving may be the deletion or simplification of objects included in captured images. The deletion of an object means the deletion of an object from an image, and the simplification may be, for example, the deletion of only the outline or the deletion of a pattern, etc., in the case of a building. The object may be a building, a feature, a vehicle, or a person that is not related to the driving of the vehicle.In view of the above, the illustrated image may be, for example, an image in which the road dividing lines are clearer than in the captured image, an image in which the lines and letters of road signs are clearer than in the captured image, an image in which only the outline lines of surrounding buildings are shown than in the captured image, an image in which the road contour lines are clearer than in the captured image (for example, the lines are thicker), an image in which faint parts of dividing lines or road signs have been filled in, or an image in which vehicles and people appearing in the captured image have been deleted.

[0027] Since there is a limit to the amount of information that humans can process at any one moment, the image generating device 100 can generate an illustrated image that can assist the driver in making driving easier by clarifying information necessary for driving and deleting other information, thereby reducing the amount of information to only that which is necessary when driving.

[0028] Furthermore, the learning model 141 may be a model that learns the correspondence between a captured image, an illustrative image to be generated based on the captured image, and condition information indicating the conditions under which the illustrative image is generated. That is, the learning model 141 may be a learning model that learns a combination of a captured image, condition information for generating an illustrative image, and an illustrative image generated according to the conditions indicated by the condition information as training data. The illustrative image used as the training data in this case may also be a CG created by a person by looking at the captured image.

[0029] The condition information is information that specifies what kind of illustration is to be generated, and the information may be the angle of view of the generated illustration (information that specifies from which direction the location of the captured image is viewed), information on what to emphasize (sharpen) for the object drawn in the generated illustration, information on what to soften, or information that specifies the color tone. As an example, the condition information may be information that specifies the generation of an overhead image from an X-degree angle of depression. Here, X degrees may be various angles such as 10 degrees, 20 degrees, 45 degrees, 80 degrees, and 90 degrees, and may be any value. Therefore, as an example, the teacher data may be composed of a combination of the captured image, an overhead image from an angle of depression of 70 degrees as condition information, and an illustrated image of the location where the captured image is captured viewed from an angle of depression of 70 degrees.

[0030] Therefore, by inputting a captured image and condition information, the learning model 141 can generate an illustrative image from the newly input captured image in a manner similar to the illustrative image generated based on the captured image according to similar conditions that have already been learned.

[0031] The learning model 141 may be generated by the image generating device 100, or may be generated by an external device different from the image generating device 100.

[0032] The learning model 141 may be stored in the memory unit 104 after being obtained from an external device via the communication unit 101, or may be stored in a flash memory or the like via the input unit 102.

[0033] CPU 105 is a processor that executes processes to be executed by image generating device 100, using various programs and various data stored in storage unit 104. That is, CPU 105 uses a learning model stored in storage unit 104 to generate an illustration image from the input captured image and condition information.

[0034] The CPU 105 may function as an acquisition unit, a reception unit, and a generation unit. The CPU 105 may also function as a learning unit.

[0035] The acquisition unit may acquire an image captured from the vehicle as the basic image. For example, the acquisition unit may acquire an image captured outside the vehicle from an external device such as the vehicle or a camera (imaging unit) mounted on the vehicle via the communication unit 101. Alternatively, for example, the acquisition unit may acquire an image captured from the vehicle including a road as the basic image, which is stored in a storage medium such as a flash memory connected to the image generating device 100, via the input unit 102. The acquisition unit transmits the acquired basic image to the generating unit.

[0036] The reception unit receives input of generation conditions for generating an illustrative image. For example, the reception unit may receive condition information indicating the generation conditions for generating an illustrative image that is received by the communication unit 101 and transmitted from an external device. Alternatively, for example, the reception unit may receive input of information indicating the generation conditions directly from an operator of the image generating device 100 via the input unit 102. The reception unit transmits the received generation conditions to the generation unit.

[0037] The receiving unit does not have to receive the generation conditions for generating the illustrative image. In this case, the generating unit described below may generate the illustrative image according to a predetermined standard.

[0038] The generating unit generates an illustrative image from the basic image. When the generating conditions are transmitted from the receiving unit, the generating unit generates an illustrative image based on the basic image transmitted from the acquiring unit, according to the generating conditions. The basic image is a real-life image captured by a camera, whereas the illustrative image is an image generated by the generating unit, and may be an image obtained by converting an image of a location indicated by the captured image into CG. The illustrative image generated based on the basic image is an image of the same location as the location indicated by the basic image. The generating unit may generate the illustrative image from the basic image so that the vehicle's situation, the road on which the vehicle is traveling, and the surrounding road environment are clear to a driver driving the vehicle.

[0039] The generating unit inputs the transmitted base image and, if the generating conditions have been transmitted from the receiving unit, the generating conditions into the learning model 141, and generates an illustration image.

[0040] The generating unit may be a generating AI of a type that generates images. As an example, the generating unit may generate an illustration image based on a captured image by using a Generative Adversarial Network (GAN), a Variational Auto Encoder (VAE), a Convolutional Neural Network (CNN), or the like. Note that these are merely examples and are not intended to be limiting.

[0041] The generation unit may cause the generated illustration image to be output by output unit 103.

[0042] The learning unit has a function of generating a learning model 141.

[0043] The learning unit uses a pair of a captured image and an illustrative image to be generated based on the captured image as teacher data, learns a correspondence relationship between the illustrative image to be generated based on the captured image, and generates a learning model 141. The learning unit may use a captured image, an illustrative image to be generated based on the captured image, and condition information for generating the illustrative image as teacher data, learns a correspondence relationship between the captured image and the illustrative image to be generated based on the condition information, and generates the learning model 141.

[0044] As an example, the learning unit may generate the learning model 141 using various types of deep learning such as GAN, VAE, SNN, support vector machine, and neural network, but is not limited to these.

[0045] The above is an example of the configuration of the image generating device 100.

[0046] <Example of image generation device operation> Hereinafter, a description will be given of the operation of the image generating device 100. First, the operation of the image generating device 100 when generating the learning model 141 will be described with reference to FIG.

[0047] 2, the CPU 105 of the image generating device 100 accepts a captured image, an illustrated image to be created based on the captured image, and generation conditions for generating the illustrated image from the captured image as teacher data in association with each other (step S201). The CPU 105 may receive the teacher data by communication from an external device via the communication unit 101, for example, or may accept teacher data based on an input from an operator of the image generating device 100 via the input unit 102, or teacher data stored in a flash memory connected to the input unit 102.

[0048] The CPU 105 learns the received multiple pieces of training data, learns an illustration image to be generated based on the captured image and the generation conditions, and generates a learning model (step S202). That is, the CPU 105 learns the relationship between the captured image and the generation conditions, and the illustration image.

[0049] The CPU 105 stores the generated learning model in the storage unit 104 (step S203). The learning model 141 stored in the storage unit 104 is used to generate an illustration image from a captured image.

[0050] Next, the process performed by the image generating device 100 when generating an illustration image based on a captured image will be described with reference to FIG.

[0051] The acquisition unit of the CPU 105 receives a captured image (step S301). The CPU 105 may receive the captured image from an external device through communication via the communication unit 101, or may receive the captured image from a flash memory connected to the image generating device 100 through the input unit 102.

[0052] The reception unit of CPU 105 receives input of generation conditions for generating an illustration image (step S302). CPU 105 may receive the generation conditions by communication from an external device via communication unit 101, or may receive the input of the generation conditions from an operator of image generating device 100 via an input unit.

[0053] The generation unit of CPU 105 inputs the captured image and the generation conditions to learning model 141 (step S303). As a result, CPU 105 generates an illustration image according to the generation conditions based on the captured image (step S304).

[0054] The generating unit of the CPU 105 outputs the generated illustrative image to the output unit 103 (step S305), and ends the process. The output of the illustrative image may be realized by transmitting the illustrative image to an external device via the communication unit 101, or may be output by displaying it on a monitor connected to the image generating device 100. The illustrative image generated in this manner may be used during navigation by a navigation device. In this manner, the image generating device 100 generates an illustrative image based on a captured image. Since the image generating device 100 can automatically generate an illustrative image from a captured image, it is possible to reduce the effort and time required to generate an illustrative image.

[0055] <Example of illustration image> Here, an example of an illustrative image generated by the image generating device 100 will be described with reference to the drawings. FIG. 4 is a diagram showing a captured image and a first example of an illustrative image generated based on the captured image. FIG. 4(a) is an example of a captured image 401, and FIG. 4(b) is an example of a generated illustrative image 402. The illustrative image 402 shown in FIG. 4(b) is an image obtained by converting the captured image 401 shown in FIG. 4(a) into an illustration with the same angle of view, as shown in the figure. As can be understood by comparing FIG. 4(b) with FIG. 4(a), the illustrative image generated by the image generating device 100 based on the captured image is an image in which information related to roads and vehicles is made clear and other information is made blurred (smooth). Specifically, the road demarcation line 411, which is blurred in FIG. 4(a), is displayed as a thicker line and the number of demarcation lines 421 in FIG. 4(b) is increased compared to FIG. 4(a), and it can be said that the existence and position of the demarcation lines are made clearer. Also, in Fig. 4(a), the road sign 412 is blurred, whereas in Fig. 4(b), the road sign 422 is supplemented and clarified, so it can be said to be clarified. Also, the fence shown in Fig. 4(a) is deleted in Fig. 4(b), and the apartment building 413 in Fig. 4(a) is blurred (smoothed) to the extent that it is possible to understand that there is a building 423 there, and information has been removed.

[0056] 5A and 5B are diagrams showing a second example of a captured image and an illustrative image generated based on the captured image, in which (a) of Fig. 5 is an example of a captured image 501, and (b) of Fig. 5 is an example of an illustrative image 502 generated.

[0057] As can be seen by comparing Fig. 5(a) with Fig. 5(b), information such as lane markings is made clearer in the illustrated image. Furthermore, Fig. 5(b) shows an illustrated image in which conversion to an overhead image is further specified as a generation condition for the captured image shown in Fig. 5(a). As a result, the illustrated image 502 shown in Fig. 5(b) is an image that allows the viewer to understand the road shape and number of lanes when turning right or left, which are blind spots and cannot be understood in the captured image 501. Note that Fig. 5(a) is an image captured at point 520 in Fig. 5(b).

[0058] The captured image and the illustration image shown in FIG. 4 and FIG. 5 may be an image pair used as training data.

[0059] <Summary> As described above, the image generating device 100 can automatically generate an illustrated image from a captured image. Therefore, it can be said that the image generating device 100 can reduce the time and effort that was conventionally required to generate a base image, generate an image to be used in a navigation device based on the generated base image, and calibrate the image. By illustrating the captured image, information necessary for the driver is made clear, and unnecessary information can be eliminated to reduce the amount of information. Therefore, the image generating device 100 can generate an image to be used by a navigation device that can provide an image that is helpful when driving.

[0060] <Supplementary Information> The image generating device according to the above embodiment is not limited to the above embodiment, and may be realized by other methods. Various modifications will be described below.

[0061] (1) In the above embodiment, the image generating device 100 may be realized as a device that generates an image used by a navigation device provided in a vehicle when providing guidance. That is, the illustrative image generated by the image generating device 100 shown in the above embodiment may be registered in map information in association with a position indicated by a location where a captured image that is the source of the illustrative image was captured and a corresponding position in map information used for navigation. Then, when the vehicle approaches the position, the navigation device may display the associated illustrative image on the display screen.

[0062] Furthermore, the illustration images may be generated only for locations that are helpful in driving a vehicle, particularly when the road structure is complicated, and may be associated with the map information.

[0063] The navigation device may be provided in a vehicle or the like, and may acquire vehicle position information using GNSS, etc. Then, when the vehicle position information reaches a position in map information to which an illustrative image generated by the image generating device 100 is associated (or when the vehicle position information comes within a predetermined range of the associated position), the navigation device may display the associated illustrative image on a display screen of the navigation device.

[0064] The navigation device may have a function of receiving input of a starting point (or a current location) and a destination, searching for a route, and providing guidance on a route to the destination.

[0065] (2) In the above embodiment, the illustrative image generating function realized by using the learning model 141 may be realized by a plurality of learning models. That is, the function realized by the learning model 141 may be realized by a first learning model for converting a captured image into an illustrative image and a second learning model for converting the generated illustrative image so as to further match the conditions indicated by the condition information. The first learning model is a model that has learned the correspondence between the captured image and the illustrative image to be generated based on the captured image. In addition, the second learning model is a model that has learned the correspondence between the illustrative image generated by the first learning model and the illustrative image generated according to the condition information.

[0066] In this case, if the receiving unit does not receive the condition information, the generation unit may output an illustrative image obtained by inputting the captured image into the first learning model, and if the receiving unit receives the condition information, the generation unit may output an illustrative image obtained by further inputting the illustrative image obtained by inputting the captured image into the first learning model into a second learning model.

[0067] Alternatively, the learning model 141 may be realized by three learning models: a first learning model for converting a captured image into an illustrated image; a second learning model for clarifying roads and dividing lines, clarifying road signs, and blurring and deleting surrounding objects so that the generated illustrated image can withstand the specifications of guidance for a navigation device; and a third learning model for performing conversion according to generation conditions based on condition information. In this case, the first learning model is a model that learns a pair of a captured image and an illustrated image to be generated based on the captured image as teacher data, and is a model that learns the correspondence between the captured image and the illustrated image. In addition, the second learning model is a model that learns a pair of an illustrated image and an illustrated image obtained by clarifying roads and dividing lines, clarifying road signs, and blurring and deleting surrounding objects from the illustrated image as teacher data, and is a model that learns the relationship between the illustrated image and the illustrated image obtained by clarifying roads and dividing lines, clarifying road signs, and blurring and deleting surrounding objects from the illustrated image. The third learning model is a learning model that learns from pairs of illustrated images in which roads and dividing lines have been clarified, road signs have been clarified, and surrounding objects have been blurred or removed, generation conditions, and illustrated images that should be generated based on the generation conditions as training data, and is a model that learns the relationship between the illustrated images in which roads and dividing lines have been clarified, road signs have been clarified, and surrounding objects have been blurred or removed, the generation conditions indicated by the condition information, and the illustrated images that should be generated based on the generation conditions.

[0068] (3) In the above embodiment, a part of the conversion from a captured image to an illustrated image using the generation unit and the learning model 141 may be realized by machine learning or the like without using a learning model. As an example, the process of deleting a predetermined object from a captured image may be performed by storing multiple types of images of the object to be deleted and deleting it by pattern matching or the like. In addition, the clarification of road lines and lane lines may be realized by identifying road lines and lane lines from detected edges based on edge detection from an image and performing a process of clarifying the identified lines (such as thickening the line segments or darkening the color).

[0069] (4) In the above embodiment, the image generating device 100 generates one illustrated image from one captured image, but this is not limited to the above. When generating an illustrated image, a plurality of captured images may be referenced. That is, one illustrated image may be generated from a plurality of captured images. Depending on the generation conditions, a desired illustrated image may not be generated with only one captured image. As a specific example, while the captured image is an image from the vehicle's perspective, if the generation conditions are an overhead view at a predetermined angle, the generated illustrated image is an overhead view image, but there are cases where it is impossible to understand what is happening due to a blind spot from the captured image. For example, at a location where a vehicle is approaching an intersection, the right-turn lane and the left-turn lane relative to the vehicle's traveling direction cannot be recognized from the vehicle position. Therefore, the image generating device 100 may use not only captured images captured from the vehicle, but also captured images captured at the same location as the captured image but from a different angle from the captured image from the vehicle. In such a case, the learning model 141 is a model that learns from data pairs of multiple captured images, condition information, and an illustration image to be created based on the multiple captured images as teacher data, and is a model that learns the correspondence between the multiple captured images, the condition information, and the illustration image to be created based on the multiple captured images.

[0070] (5) In the above embodiment, the method of generating an illustration image from a captured image in the image generating device is specified by the processor of the image generating device executing a predetermined program or the like. However, this may be realized by a logic circuit (hardware) or a dedicated circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or the like in the device. Moreover, these circuits may be realized by one or more integrated circuits, and the functions of the multiple functional units shown in the above embodiment may be realized by one integrated circuit. The LSI may be called a VLSI, a super LSI, an ultra LSI, or the like depending on the degree of integration. That is, as shown in FIG. 6, the image generating device 100 may be composed of a communication circuit 101a, an input circuit 102a, an output circuit 103a, a memory circuit 104a, and a control circuit 105a, which correspond to the communication unit 101, the input unit 102, the output unit 103, the memory unit 104, and the CPU 105, respectively.

[0071] The program may be recorded on a processor-readable recording medium, and the recording medium may be a "non-transient tangible medium" such as a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The program may be supplied to the processor via any transmission medium capable of transmitting the program (such as a communication network or a broadcast wave). That is, for example, the program may be downloaded from a network and executed using an information processing device such as a smartphone. The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0072] The above program can be implemented using, for example, scripting languages ​​such as ActionScript and JavaScript (registered trademark), and object-oriented programming languages ​​such as Objective-C, Java (registered trademark), C++, Python, and R, but these languages ​​are just examples.

[0073] (6) The various examples shown in the above embodiment may be combined as appropriate. Furthermore, the operations shown in the flowcharts may be executed in a different order or in parallel if no contradiction results. For example, the processing of steps S301 and S302 may be executed in the reverse order, i.e., steps S302 and S301, or may be executed simultaneously. [Explanation of symbols]

[0074] 100 Image generating device 101 Communications Department 102 Input section 103 Output section 104 Storage section 105 CPU (acquisition unit, reception unit, generation unit, learning unit) 141 Learning Model

Claims

1. a storage unit that stores a learning model that learns a relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions; An acquisition unit that acquires a basic image captured from a vehicle; a reception unit that receives an input of generation conditions for generating an illustration image; A generation unit that inputs the base image and the generation conditions into the learning model and generates a generated illustration image based on the base image in accordance with the generation conditions; An image generating device comprising:

2. The generating unit generates the generated illustration image based on the captured image so as to clarify a road.

2. The image generating device according to claim 1.

3. The generating unit generates the generated illustrated image so as to clarify road markings of the road.

3. The image generating apparatus according to claim 2.

4. The generating unit generates the generated illustrated image so as to simplify features other than roads.

3. The image generating apparatus according to claim 2.

5. The generating unit generates the generated illustration image by deleting a predetermined object included in the captured image based on the captured image.

2. The image generating device according to claim 1.

6. The image generating apparatus according to any one of claims 1 to 5, further comprising an output section for outputting the generated illustrative image.

7. A storage unit that stores registered map information for a generated illustrative image generated by the image generating device according to any one of claims 1 to 5, based on a position where a captured image used in generating the generated illustrative image was captured; An acquisition unit for acquiring location information; an output unit that outputs the generated illustrated image corresponding to a position when the position information indicates a position that satisfies a predetermined condition; A navigation device comprising:

8. The computer a storage step of storing a learning model that has learned a relationship between a captured image taken from a vehicle, conditions for generating an illustrative image based on the captured image, and the captured image and the illustrative image based on the conditions; An acquisition step of acquiring a basic image captured from a vehicle; A receiving step of receiving an input of generation conditions for generating an illustration image; A generation step of inputting the base image and the generation conditions into the learning model to generate a generated illustration image based on the base image in accordance with the generation conditions; An image generation method that performs the following:

9. On the computer, a storage function for storing a learning model that has learned a relationship between an image captured from a vehicle, conditions for generating an illustrative image based on the captured image, and the illustrative image based on the conditions; and An acquisition function for acquiring a basic image captured from a vehicle; A reception function for receiving input of generation conditions for generating an illustration image; A generation function of inputting the base image and the generation conditions into the learning model and generating a generated illustration image based on the base image in accordance with the generation conditions; An image generation program that realizes this.

Images