Image processing device, image processing method, and recording medium

Abstract

An image processing device according to the present disclosure has an acquisition unit, a compositing unit, and a processing unit. The acquisition unit acquires an external image obtained by photographing the outside of a moving body and an internal image obtained by photographing the inside of the moving body. The compositing unit generates a composite image in which the external image and the internal image are composited. The processing unit generates a processed composite image in which a specific region in the composite image has a higher image quality than another region.

Description

Image Processing Apparatus, Image Processing Method, and Recording Medium

[0001] This disclosure relates to an image processing apparatus, an image processing method, a recording medium, and a program.

[0002] A technology related to this disclosure is disclosed in Patent Document 1. Patent Document 1 discloses acquiring images of the front, rear, left, and right sides of a vehicle, converting the images in each of a plurality of directions into a predetermined bird's-eye view image, and generating a composite image by synthesizing the bird's-eye view images in the plurality of directions. Further, Patent Document 1 discloses identifying an area including a target to be alerted in an image and processing the image so that the resolution of the identified area is higher than that of other areas.

[0003] Japanese Unexamined Patent Application Publication No. 2010-283567

[0004] In recent years, cameras have been installed in many moving bodies, and various processes based on images generated by the cameras have been performed. An example of the purpose of this disclosure is to develop a technology related to images generated by a camera installed in a moving body.

[0005] According to one aspect of this disclosure, there is provided an image processing apparatus including: acquisition means for acquiring an external image obtained by photographing the outside of a moving body and an internal image obtained by photographing the inside of the moving body; synthesis means for generating a composite image by synthesizing the external image and the internal image; and processing means for generating a processed composite image in which a specific area in the composite image has higher image quality than other areas.

[0006] Further, according to one aspect of this disclosure, there is provided an image processing method in which one or more computers acquire an external image obtained by photographing the outside of a moving body and an internal image obtained by photographing the inside of the moving body, generate a composite image by synthesizing the external image and the internal image, and generate a processed composite image in which a specific area in the composite image has higher image quality than other areas.

[0007] Furthermore, according to one aspect of this disclosure, a recording medium is provided for recording a program that causes a computer to function as: an acquisition means for acquiring external images of the outside of a moving object and internal images of the inside of a moving object; a synthesis means for generating a composite image by combining the external image and the internal image; and a processing means for generating a processed composite image in which a specific region within the composite image has higher image quality than other regions.

[0008] One example of this disclosure suggests that technologies related to images generated by cameras installed on mobile devices can be advanced.

[0009] Figure 1 is a diagram showing an example of a functional block diagram of an image processing device. Figure 2 is a flowchart showing an example of the processing flow of an image processing device. Figure 3 is a diagram showing an example of the hardware configuration of an image processing device. Figure 4 is a diagram showing another example of a functional block diagram of an image processing device. Figure 5 is a diagram showing an example of a composite image generated by an image processing device. Figure 6 is a flowchart showing another example of the processing flow of an image processing device. Figure 7 is a flowchart showing another example of the processing flow of an image processing device. Figure 8 is a diagram showing another example of a functional block diagram of an image processing device.

[0010] The embodiments of this disclosure will be described below with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0011] <<First Embodiment>> Figure 1 is a functional block diagram showing an overview of the image processing device 10. Figure 2 is a flowchart showing an example of the processing flow executed by the image processing device 10.

[0012] As shown in Figure 1, the image processing apparatus 10 includes an acquisition unit 11, a synthesis unit 12, and a processing unit 13. These functional units execute the processes shown in the flowchart of Figure 2.

[0013] In S10, the acquisition unit 11 acquires an external image taken of the outside of the moving object and an internal image taken of the inside of the moving object. In S11, the synthesis unit 12 generates a composite image by combining the external image and the internal image. In S12, the processing unit 13 generates a processed composite image in which a specific area within the composite image has higher image quality than other areas.

[0014] In this way, the image processing device 10 generates a composite image by combining multiple images (external images and internal images). In one example of the use of external and internal images of a mobile object, these multiple images are transmitted to an external device and processed outside the mobile object. For example, multiple images may be displayed on a screen at a processing center located outside the mobile object, and a supervisor may monitor these multiple images. In another case, multiple images may be displayed on a screen at a processing center located outside the mobile object, and a remote operator may remotely control the mobile object while viewing these multiple images. In yet another case, a computer remotely controlling the mobile object may analyze these multiple images to determine the control content of the mobile object.

[0015] In this example, multiple images (external and internal images) are transmitted to an external device. However, if multiple images are sent and received separately, there is a risk of time lags occurring between the images due to transmission / reception processing and post-reception processing. For example, when multiple images are displayed simultaneously on a display at a processing center, multiple images captured at the same time may not be displayed at the same time, potentially resulting in a time lag.

[0016] In contrast, the image processing device 10, which generates a composite image by combining multiple images, allows multiple images to be sent and received together by sending and receiving the composite image. The processing center can then display the composite image, enabling the simultaneous display of multiple images. Such an image processing device 10 can suppress the problem of time delays occurring between multiple images due to transmission / reception processing or post-reception processing.

[0017] Furthermore, the image processing device 10 generates a processed composite image in which a specific region within the composite image has higher image quality than other regions. When transmitting and receiving a composite image created by combining multiple images as described above, the bitrate increases, and the communication burden becomes greater. A processed composite image in which a specific region within the composite image has higher image quality (higher resolution) than other regions is, in other words, an image in which some regions within the composite image have lower image quality (lower resolution) than other regions. Such a processed composite image can reduce the bitrate compared to a composite image in which all regions have high image quality.

[0018] Furthermore, the bitrate can be reduced even further by making all areas of the composite image low quality. However, if the image quality is low, the visibility of the composite image may decrease, and problems may arise in that the status of moving objects cannot be accurately grasped. Therefore, the image processing device 10 does not make all areas of the composite image low quality, but rather makes some areas low quality and other areas high quality. With the image processing device 10 that can make some areas of the composite image low quality and other areas high quality in this way, it is possible to make the parts of the composite image that are relatively important high quality and parts that are relatively less important low quality. With such an image processing device 10, it is possible to ensure sufficient visibility by making the important parts high quality, while reducing the bitrate to a certain extent by making other parts low quality. In other words, the image processing device 10 can achieve a good balance between good visibility in the composite image and a reduction in bitrate.

[0019] Furthermore, the image processing device 10 generates a composite image by combining an external image taken of the outside of the moving object and an internal image taken of the inside of the moving object. The technology disclosed in Patent Document 1 generates a composite image by combining external images taken of the outside of the moving object. In the case of the technology disclosed in Patent Document 1, the external situation of the moving object can be understood based on the composite image, but the internal situation of the moving object cannot be understood. In contrast, with the image processing device 10, it is possible to understand not only the external situation of the moving object but also the internal situation of the moving object based on the composite image. In other words, with the image processing device 10, it is possible to understand the situation of the moving object in more detail.

[0020] <<Second Embodiment>> <Overview> The image processing apparatus 10 of the second embodiment is a concrete implementation of the configuration of the image processing apparatus 10 of the first embodiment. It will be described in detail below.

[0021] <Hardware Configuration> First, an example of the hardware configuration of the image processing device 10 will be described. Each functional unit of the image processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various modifications to the implementation method and the device. The software includes programs that are pre-installed at the time of shipment of the device, as well as programs downloaded from recording media such as CDs (Compact Discs) or from servers on the Internet.

[0022] Figure 3 is a block diagram illustrating the hardware configuration of the image processing device 10. As shown in Figure 3, the image processing device 10 includes a processor 1A, memory 2A, input / output interface 3A, peripheral circuitry 4A, and bus 5A. The peripheral circuitry 4A includes various modules. The image processing device 10 does not necessarily have peripheral circuitry 4A. The image processing device 10 may also be composed of multiple physically and / or logically separated devices. In this case, each of the multiple devices may have the above hardware configuration.

[0023] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to send and receive data to and from each other. The processor 1A is a processing unit such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit). Memory 2A is a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes interfaces for connecting to a communication network such as the Internet. Input devices include, for example, a keyboard, mouse, microphone, physical buttons, touch panel, etc. Output devices include, for example, a display, projection device, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on their calculation results.

[0024] <Functional Configuration> Next, the functional configuration of the image processing device 10 will be described in detail. Figure 4 is an example of a functional block diagram of the image processing device 10. As shown in the figure, the image processing device 10 has an acquisition unit 11, a synthesis unit 12, a processing unit 13, and a transmission unit 14.

[0025] The acquisition unit 11 acquires external images of the outside of the mobile body and internal images of the inside of the mobile body. The image processing device 10 is mounted on the mobile body. The acquisition unit 11 acquires external images of the outside of the mobile body on which it is mounted and internal images of the inside of the mobile body. The acquisition unit 11 can acquire one or more external images. The acquisition unit 11 can also acquire one or more internal images.

[0026] A "mobile object" is an object that has the function of moving. A mobile object may also have the function of transporting people and goods. A mobile object may be an object that moves on land (bus, train, truck, four-wheeled vehicle, two-wheeled vehicle, work machine, etc.). In addition, a mobile object may be an object that moves on water (ship, yacht, boat, etc.). In addition, a mobile object may be an object that moves underwater (submarine, etc.). In addition, a mobile object may be an object that moves in the air (airplane, helicopter, drone, etc.). Note that the mobile objects exemplified here are merely examples and are not limited to these.

[0027] The mobile device may be equipped with an unmanned operation function. In this case, the mobile device may be equipped with a function to move according to the control of a computer installed on the mobile device. Alternatively, the mobile device may be equipped with a function to move according to the control of a computer located outside the mobile device. Furthermore, the mobile device may be equipped with a function to move according to remote control by a driver (person) located outside the mobile device without the driver being on board. The mobile device may also be equipped with a manned operation function, which allows it to move under the control of a driver on board the mobile device. Furthermore, the mobile device may be equipped with both an unmanned operation function and a manned operation function.

[0028] The mobile device is equipped with multiple cameras. The cameras may detect and image visible light, or detect and image infrared light or other electromagnetic waves. The cameras may also image data measured by LiDAR (Light Detection and Ranging). The cameras may capture moving images or still images. The cameras may be monocular or stereo cameras. The cameras may be equipped with a standard lens with a field of view of about 45°, a wide-angle lens with a field of view of about 63°, a fisheye lens with a field of view of 180° or more, or other types of lenses. Multiple cameras installed on a single mobile device may have different functions / performance characteristics.

[0029] Multiple cameras are installed in positions and orientations that capture predetermined locations on the moving object. One or more of the multiple cameras are installed in positions and orientations that capture the outside of the moving object (around the moving object). For example, multiple cameras may be installed to capture the front, rear, right side, left side, and near the doors of the moving object, respectively, as cameras to capture the outside of the moving object. In addition, cameras may be installed to capture other directions of the moving object, such as the front right, front left, rear right, rear left, and above the moving object. Furthermore, cameras may be installed to capture predetermined parts of the moving object located on the outside, such as tires, mufflers, turn signals, wipers, headlights, taillights, and side mirrors.

[0030] Furthermore, one or more of the cameras are positioned and oriented to photograph the interior of the mobile vehicle. The interior of the mobile vehicle includes at least one of the following: a room where a person is located, a space where objects are stored (trunk room, cargo bed, etc.), and a space where parts of the mobile vehicle (engine, etc.) are stored (engine room, etc.).

[0031] For example, the camera may be installed in a position and orientation to photograph the driver located in the driver's seat. The camera may also be installed in a position and orientation to photograph people located in other seats of the moving vehicle (such as the passenger seat or rear seats). Furthermore, in the case of a moving vehicle that has a space where passengers are located, such as a bus, train, or airplane, the camera may be installed in a position and orientation to photograph the space where the passengers are located. The camera may also be installed in a position and orientation to photograph an information display device installed on the moving vehicle. The information display device is, but is not limited to, a display or an instrument panel. The camera may also be installed in a position and orientation to photograph an object located in a space where objects are housed. Furthermore, the camera may be installed in a position and orientation to photograph a predetermined part located in a space where parts of the moving vehicle are housed.

[0032] The image processing device 10 is communicatively connected to a camera installed on a mobile device. The acquisition unit 11 can acquire images generated by such a camera installed on a mobile device in real time.

[0033] The compositing unit 12 generates a composite image by combining external and internal images. As shown in Figure 5, the compositing unit 12 arranges external and internal images captured at the same time in a predetermined layout to generate a single image (composited image). The composite image includes one or more external images. The composite image also includes one or more internal images. When multiple cameras capture moving images, the compositing unit 12 arranges multiple frame images captured at the same time as shown in Figure 5 to generate a composite image.

[0034] For example, as shown in Figure 5, the placement position and size of images generated by each of the multiple cameras within the composite image are predetermined. The user may change these placement positions and sizes. The compositing unit 12 can generate a composite image like the one shown in Figure 5 by placing the images generated by each of the multiple cameras at predetermined placement positions and sizes within the composite image. In Figure 5, six images are arranged to generate one composite image, but the number of images to be composited is not limited to the example shown. Also, in the example in Figure 5, the six external and internal images are arranged in two vertical columns and three horizontal columns, but the arrangement is not limited to this example. Furthermore, in the example in Figure 5, all six external and internal images are the same size, but this is not limited to this, and the sizes of the multiple external and internal images in the composite image may differ from each other.

[0035] Returning to Figure 4, the processing unit 13 performs predetermined processing on the composite image generated by the synthesis unit 12 to generate a processed composite image. The processing unit 13 generates a processed composite image in which a specific region within the composite image has higher image quality than other regions.

[0036] A "specific region" is a part of the composite image that has relatively high importance. In the processed composite image, the image quality of such specific regions becomes relatively high (high quality). Conversely, in the processed composite image, the image quality of regions other than the specific region, which have relatively low importance, becomes relatively low (low quality). Thus, the processed composite image contains both regions with relatively high image quality and regions with relatively low image quality.

[0037] Here, we will explain the process of determining the region (specific region) in the composite image in which the image quality will be improved. In the second embodiment, the processing unit 13 can determine a portion of the composite image specified by user input as the specific region. Hereafter, the user input specifying a portion of the composite image may be referred to as the first user input. In the following embodiment, we will explain a configuration in which the computer automatically determines the specific region.

[0038] In one example, the driver riding in the mobile device may perform the first user input. In this case, the driver can perform the first user input via the input device provided by the image processing device 10. Alternatively, the first user input may be performed by a monitor who monitors the status of the mobile device based on the image displayed on the processing center's display as described in the first embodiment, or by a remote operator who remotely controls the mobile device based on the image. In this case, the monitor or remote operator can perform the first user input via an input device installed in the processing center. The content of the first user input is then transmitted from the system installed in the processing center to the image processing device 10.

[0039] The first user input specifies a portion of the composite image. The processing unit 13 then determines the portion of the composite image specified by the first user input as a specific region. A configuration that accepts a first user input specifying a portion of the composite image can be realized using various technologies. An example is described below, but it is not limited to this example.

[0040] For example, the first user input may specify a portion of the composite image based on the images generated by cameras installed on the moving object. In the example shown in Figure 5, at least one of the following is selected in the first user input: "in-cabin camera," "door-side camera," "rear camera," "left-side camera," "front camera," and "right-side camera." In this case, the region in the composite image where images generated by the selected camera exist is specified in the first user input and becomes the specific region.

[0041] Although not shown in FIG. 5, a frame may be superimposed on the composite image and displayed. Then, with the first user input, the position, size, shape, etc. of the frame displayed on the composite image may be changed. In the case of this example, the area within the frame is specified by the first user input and becomes a specific area.

[0042] Next, a process of generating a processed composite image in which a specific area within the composite image has higher image quality than other areas will be described.

[0043] The processing unit 13 generates a processed composite image by compression-encoding the composite image using a codec corresponding to the ROI (Region of Interest). According to the ROI function, the specified area (specific area) can be made to have higher image quality while maintaining the specified bandwidth (that is, the other areas are made to have lower image quality). Examples of the codec corresponding to the ROI include, but are not limited to, H.264 SVC (Scalable Video Coding), H.265 / MPEG-H HEVC, etc.

[0044] Returning to FIG. 4, the transmission unit 14 transmits the processed composite image to an external device. The transmission unit 14 can transmit the processed composite image to the external device via a communication network such as the Internet.

[0045] The external device can perform various processes on the received processed composite image. For example, the external device may execute any one of the following Processes 1 to 3. Note that the external device may perform other processes.

[0046] "Process 1" In Process 1, the external device decodes the processed composite image and displays the decoded composite image on the display. As a result, as shown in FIG. 5, one image obtained by synthesizing the images generated by a plurality of cameras is displayed on the display.

[0047] The monitor can grasp the situation of the moving body by visually recognizing the composite image displayed on the display.

[0048] "Process 2" In Process 2, the external device constitutes a remote operation system for remotely operating the mobile body. The external device decrypts the processed composite image and displays the decrypted composite image on the display. As a result, as shown in FIG. 5, one image obtained by synthesizing the images generated by a plurality of cameras is displayed on the display.

[0049] A remote operator who remotely operates the mobile body can grasp the situation of the mobile body by visually recognizing the composite image displayed on the display. Then, based on the grasped content, the remote operator can remotely operate the mobile body by operating the remote operation system (external device).

[0050] The "remote operation system" is a system for remotely operating the mobile body. The configuration of the remote operation system is not limited, and a configuration adopting widely known technologies can be used. For example, the remote operation system includes a communication unit that communicates with the system of the mobile body, an output unit that outputs information to the remote operator, and an input unit that receives a driving operation from the remote operator.

[0051] The communication unit can acquire data of images and sounds indicating the situation around and inside the mobile body generated by cameras and microphones installed in the mobile body from the system of the mobile body (image processing device 10). Further, the communication unit can transmit information indicating the content of the driving operation performed by the remote operator on the input unit (information indicating the states of various operation devices and the input content to various operation devices, etc.) to the system of the mobile body. Further, the communication unit may be able to acquire various information input by passengers on the mobile body to the system of the mobile body.

[0052] The output unit can, for example, process the processed composite image and sound data received by the communication unit from the system of the mobile body, and display an image via an output device such as a display or a projection device, or output sound via a speaker. The remote operator can grasp the situation around and inside the mobile body based on the displayed image and the output sound.

[0053] The input unit receives driving commands from a remote operator via an operating device. The operating device can be, for example, the same as an operating device mounted on a mobile vehicle. The operating device may include, for example, at least one of the following: a steering wheel, brake pedal, accelerator pedal, clutch pedal, handbrake, turn signal, operating lever, steering wheel, throttle lever, joystick, or touch panel. The operating device may also be any other device. The input unit may receive commands from a remote operator to apply physical force to set the operating device to a predetermined state. In addition, the input unit may receive electrical signals from a remote operator to set the operating device to a predetermined state. These electrical signals include information specifying the predetermined state of the operating device (e.g., steering wheel rotation amount, pedal depression amount, handbrake state, turn signal state).

[0054] Information indicating the details of the driving operations performed by the remote operator on the input unit (information indicating the status of various control devices and the input content to various control devices, etc.) is transmitted to the mobile vehicle's system via the communication unit. The mobile vehicle's system controls the mobile vehicle based on the received information. For example, the mobile vehicle's system can set the status of various control devices mounted on the mobile vehicle to the status of the various control devices indicated by the information received from the remote control system.

[0055] "Process 3" In Process 3, the external device decodes the processed composite image and inputs the decoded composite image into the Autonomous Driving AI (Artificial Intelligence). The Autonomous Driving AI analyzes the input composite image to understand the surrounding and internal conditions of the moving object. Based on the surrounding and internal conditions of the moving object, the Autonomous Driving AI determines the control content of the moving object and transmits information indicating the determined control content to the moving object's system. The moving object's system controls the moving object based on the received information. For example, the moving object's system can set the state of various control devices mounted on the moving object to the state of the various control devices indicated by the information received from the Autonomous Driving AI. The Autonomous Driving AI can be realized using widely known technologies.

[0056] Next, an example of the processing flow of the image processing device 10 will be explained using the flowchart in Figure 6. Note that the purpose here is to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0057] In S20, the image processing device 10 acquires images generated by a camera installed on a mobile body on which the device is mounted, including an external image capturing the outside of the mobile body and an internal image capturing the inside of the mobile body.

[0058] In S21, the image processing device 10 generates a composite image by combining the external and internal images acquired in S20, which were captured at the same time. The composite image includes one or more external images. The composite image also includes one or more internal images.

[0059] In S22, the image processing device 10 compresses and encodes the composite image generated in S21 using a codec that supports ROI, thereby generating a processed composite image. The image processing device 10 generates a processed composite image in which a specific region within the composite image has higher image quality than other regions.

[0060] For example, a specific region is pre-configured to include an area where an image generated by a designated camera (e.g., a front camera) is captured. The front camera is a camera that photographs the area in front of the vehicle. This setting can be changed by user input. The user can set an area where an image generated by another camera is captured as a specific region. The user may also set any area within the composite image as a specific region. The image processing device 10 determines the specific region according to the settings when processing the composite image.

[0061] In S23, the image processing device 10 transmits the processed composite image to an external device.

[0062] The image processing device 10 can repeat the above process. Furthermore, the image processing device 10 can perform the above process in real time.

[0063] <Effects and Effects> The image processing apparatus 10 of the second embodiment can achieve the same effects and effects as the image processing apparatus 10 of the first embodiment.

[0064] Furthermore, the image processing device 10 can generate a composite image that includes an internal image taken of the interior of a mobile body, which includes at least one of the following: a room where a person is located, a space where objects are housed, and a space where parts of the mobile body are housed.

[0065] Internal images may include, for example, images of the driver in the driver's seat or people in other seats of the vehicle (passenger seat, rear seat, etc.). Internal images may also include images of the passenger compartment of a vehicle equipped with a passenger compartment, such as a bus, train, or airplane. Furthermore, internal images may include images of information display devices installed in the vehicle, such as displays or instrument panels. Internal images may also include images of the storage compartment (trunk, cargo area, etc.) of the vehicle. Finally, internal images may include images of the storage compartment (engine compartment, etc.) of the vehicle's components.

[0066] According to the image processing device 10, which generates a composite image including such internal images and transmits it to an external device, it becomes possible to understand the status of the moving object in more detail based on the composite image.

[0067] Furthermore, the image processing device 10 can generate a composite image that includes external images of predetermined parts of the moving object that are located outside the moving object, such as tires, mufflers, turn signals, wipers, headlights, taillights, and side mirrors. By generating such a composite image that includes external images and transmitting it to an external device, the image processing device 10 makes it possible to understand the status of the moving object in more detail based on the composite image.

[0068] Furthermore, the image processing device 10 can determine, through user input, a specific region within the composite image that should have higher image quality than other regions. With such an image processing device 10, the user can designate a region within the composite image that they want to examine in more detail as a specific region, thereby making that region relatively higher quality.

[0069] <<Third Embodiment>> The image processing apparatus 10 of the third embodiment detects the state of a moving object and determines a specific region in the composite image that will have higher image quality than other regions, according to the state of the moving object. This will be explained in detail below.

[0070] Figure 4 is an example of a functional block diagram of the image processing device 10. As shown in the figure, the image processing device 10 has an acquisition unit 11, a synthesis unit 12, a processing unit 13, and a transmission unit 14.

[0071] The processing unit 13 detects the state of the moving object. Then, the processing unit 13 changes the region within the composite image that is designated as the specific region according to the state of the moving object. Specifically, based on the state of the moving object, the processing unit 13 determines that at least one of the multiple images (external image and internal image) included in the composite image is the specific image. Then, the processing unit 13 determines that the region within the composite image where the specific image exists is the specific region. Each process will be described in detail below.

[0072] First, let's explain the process for detecting the state of the moving object. The processing unit 13 can detect the state of the moving object based on at least one of the following: the position of the moving object, an external image, an internal image, and information collected by sensors mounted on the moving object.

[0073] For example, the processing unit 13 can detect the content of the information collected by sensors mounted on the mobile body as the state of the mobile body.

[0074] Examples of sensors mounted on a mobile vehicle include speed sensors, acceleration sensors, sensors that detect the open / closed state of the vehicle's doors, sensors that detect the state of the steering wheel (steering angle, etc.), sensors that detect the operating state of specific components such as turn signals and wipers, and temperature sensors. Other examples of sensors mounted on a mobile vehicle include LiDAR and radar that detect objects around the vehicle. Other examples of sensors mounted on a mobile vehicle include sensors that detect various information about the person (e.g., driver) inside the vehicle. Examples of sensors that detect various information about a person include sensors that detect the person's movement, sensors that detect the person's posture, and vital signs sensors that detect the person's biological information. Other examples of sensors mounted on a mobile vehicle include sensors that detect the vehicle's current position (e.g., GPS (global positioning system) sensors).

[0075] In addition, the processing unit 13 may analyze information collected by sensors mounted on the mobile body and detect whether or not the mobile body is in a predefined state.

[0076] The predefined states of a moving object may include, for example, at least one of the following: moving, stopped, immediately before stopping, immediately after stopping, immediately before starting, and immediately after starting. Furthermore, the states of the moving object may include states in which a predetermined event has occurred inside the moving object. Furthermore, the states of the moving object may include states in which a predetermined event has occurred outside the moving object.

[0077] "In motion" refers to a state in which the moving object is in motion. For example, the processing unit 13 may determine that the moving object is in motion if its speed is above a threshold.

[0078] "Just before stopping" refers to the state of a moving object immediately before it comes to a complete stop. For example, the processing unit 13 may determine that the state of the moving object is "just before stopping" if the speed of the moving object is below a threshold and the speed of the moving object within the immediate vicinity of a fixed time is decreasing as time progresses.

[0079] In addition, the processing unit 13 may detect that a moving object is about to stop based on its current position. This detection is particularly effective for moving objects such as buses and trains that travel sequentially to multiple points (bus stops or stations) and stop at each point for passengers to board or alight. The processing unit 13 has in advance information indicating the locations of the multiple points where the moving object will stop, and the route the moving object will take to visit those points. Based on this information, the processing unit 13 identifies the next point where the moving object will stop. The processing unit 13 may then determine that the moving object is about to stop if the distance between the moving object's current position and the next stopping position (distance along the route) is below a threshold.

[0080] "Immediately after stopping" refers to the state immediately after the moving object has come to a complete stop. For example, the processing unit 13 may determine that the state of the moving object is immediately after stopping if the elapsed time since the moving object's speed became "0" is within a predetermined time.

[0081] "Immediately before departure" refers to the state of a stationary moving object just before it starts moving. For example, the processing unit 13 may determine that the state of the moving object is "immediately before departure" if the speed of the moving object is "0" and the open / closed state of the moving object's doors changes from "open" to "closed". This determination is particularly effective for moving objects such as buses and trains that travel sequentially to multiple points (bus stops and stations) and open and close their doors at each point for passengers to get on and off.

[0082] "Stopped" means the moving object is not moving. For example, the processing unit 13 may determine that the moving object is stopped if its speed is "0" and it is not immediately after stopping or immediately before starting.

[0083] "A predetermined event inside the moving object" refers to an event that occurs inside the moving object and requires attention during its operation.

[0084] A predetermined event may be, for example, a passenger performing a predetermined action while in a moving vehicle such as a bus or train. The predetermined action may include, but is not limited to, standing up from a seat, falling, or stumbling. The processing unit 13 can analyze internal images to detect people and, based on the detected person's posture and movement, can detect the action in question. This detection can be achieved using widely known technologies.

[0085] In addition, a predetermined event may also be the driver being in an alert state. An alert state is a state in which drowsiness, fatigue, tension, etc., are above a threshold. The processing unit 13 can detect the driver's drowsiness, fatigue, tension, etc., based on images of the driver taken or the driver's biological information detected by vital sensors. This detection can be achieved using widely known technology.

[0086] Other specified events may include the tipping or falling of objects stored in the space where the moving object is housed (trunk room, cargo area, etc.). The processing unit 13 can detect the tipping or falling of objects by analyzing internal images taken of the space. This detection can be achieved using widely known technology.

[0087] In addition, the event may also be an abnormality in a component of the mobile body. For example, the processing unit 13 may detect as the event that the temperature in the space where the mobile body's components are housed (such as the engine room) exceeds a threshold. In addition, the processing unit 13 may receive an error code (such as a DTC (Diagnostic Trouble Code) code) output by the mobile body's system. The mobile body's system monitors the status of the mobile body and outputs an error code corresponding to the status when a predetermined abnormality occurs. The processing unit 13 may detect the reception of a predetermined error code as the event.

[0088] "A specified event outside the moving object" refers to an event that occurs outside the moving object and requires attention during the movement of the moving object.

[0089] A predetermined event may be the detection of a predetermined object in the vicinity of a moving object. The predetermined object may be a person, other animals, other moving objects, or other objects, but is not limited to these. The processing unit 13 may detect the predetermined object in the vicinity of the moving object by analyzing external images. Alternatively, the processing unit 13 may detect the predetermined object in the vicinity of the moving object based on sensing data such as LiDAR or radar.

[0090] Next, we will explain the process of determining at least one of the multiple images (external images and internal images) included in the composite image as a specific image based on the state of the moving object. As mentioned above, the processing unit 13 determines the region in the composite image where the specific image exists as a specific region (a region to be treated as high-quality).

[0091] The image processing device 10 has pre-registered decision rules that specify the image to be used as the designated image for each state of the moving object. When the processing unit 13 identifies the state of the moving object, it refers to the decision rules and identifies the image to be used as the designated image for the identified state of the moving object. The processing unit 13 then decides that the identified image is the designated image. An example of a decision rule is shown below.

[0092] For example, when a moving object is in motion, the forward image is designated as the specific image. The forward image is an image taken of the area in front of the moving object. When a moving object is in motion, attention must be paid to the area in front of the object.

[0093] In another example, when a moving object is in motion, its state is further subdivided based on at least one of the following: its speed, acceleration, steering wheel position, and whether it is moving forward or backward. The determination rules then specify that for each subdivided state, a portion of the front, right, left, and rear images will be designated as a specific image.

[0094] For example, if the moving object is in motion and moving forward, and the steering wheel is turned to the right by a predetermined angle or more, the front image and the right-side image may be designated as specific images. Similarly, if the moving object is in motion and moving forward, and the steering wheel is turned to the left by a predetermined angle or more, the front image and the left-side image may be designated as specific images. Furthermore, if the moving object is in motion and moving backward, the rear image may be designated as a specific image. These are merely examples and are not limited to these.

[0095] In another example, it is stipulated that when a moving object has just stopped or is stopped, the image of the area near the door should be designated as the specific image. The image of the area near the door is an image taken of the area around the door of the moving object. Immediately after or while a moving object has stopped, people may get on or off the object, so attention must be paid to the area around the door.

[0096] In another example, when a moving object is about to depart or has just departed, it is stipulated that the front image, rear image, and interior image should be designated as specific images. The front image is an image taken of the front of the moving object. The rear image is an image taken of the rear of the moving object. The interior image is an image taken of the interior of the moving object. For example, it may be stipulated that the interior image taken of the space where the passengers are located should be designated as the specific image. Just before the moving object departs, it is necessary to check the space in front of and behind the object, as well as the space where the passengers are located, and to decide whether it is OK to depart. Also, immediately after the moving object has departed, particular attention is required in the space in front of and behind the object, as well as the space where the passengers are located.

[0097] In another example, when a moving object is about to stop, the front image and the left-side image are designated as specific images. The front image is an image taken of the front of the moving object. The left-side image is an image taken of the left side of the moving object. When a moving object is in motion, attention must be paid to what is in front of it. Also, when stopping a moving object, it should be moved to the left side of the road. Therefore, attention must be paid to what is on the left side of the moving object. Note that, depending on the traffic rules, when stopping a moving object, it may be necessary to move it to the right side of the road. When using the image processing device 10 in an area with such traffic rules, when a moving object is about to stop, the front image and the right-side image are designated as specific images.

[0098] In another example, it is stipulated that if a predetermined event occurs inside a moving object, the internal image will be designated as the specific image.

[0099] For example, if a passenger performing a predetermined action inside a moving vehicle such as a bus or train is detected as the occurrence of a predetermined event, it may be stipulated that an internal image of the space where the passenger is located within the moving vehicle be designated as the specific image.

[0100] Furthermore, if the driver's state is detected as being in an alert state as a predetermined event, it may be stipulated that the internal image of the driver of the moving vehicle be designated as the specific image.

[0101] Furthermore, if the occurrence of an event is detected where an object stored in the space containing the object of the moving vehicle (trunk room, cargo area, etc.) has been toppled or fallen, it may be stipulated that an internal image of the space containing the object of the moving vehicle be designated as the specific image.

[0102] Furthermore, if an abnormality in a component of the mobile body is detected as the occurrence of a predetermined event, it may be stipulated that an internal image taken of the space where the component of the mobile body is housed (such as the engine room) shall be designated as the specific image.

[0103] In another example, if a predetermined event is detected outside the moving object, the external image taken at the location where the predetermined event occurred is designated as the specific image. In this example, the predetermined event is the detection of a predetermined object around the moving object. In this case, the processing unit 13 determines that the external image containing the detected predetermined object is the specific image.

[0104] According to the decision rules described above, an appropriate image can be determined as a specific image depending on the state of the moving object.

[0105] Next, an example of the processing flow of the image processing device 10 will be explained using the flowchart in Figure 7. Note that the purpose here is to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0106] In S30, the image processing device 10 acquires images generated by a camera installed on a mobile body on which the device is mounted, including an external image capturing the outside of the mobile body and an internal image capturing the inside of the mobile body.

[0107] In S31, the image processing device 10 generates a composite image by combining the external and internal images acquired in S30, which were captured at the same time. The composite image includes one or more external images. The composite image also includes one or more internal images.

[0108] In S32, the image processing device 10 detects the state of the mobile body on which it is mounted.

[0109] In S33, the image processing device 10 determines a specific region based on the state of the moving object detected in S32. Specifically, the image processing device 10 determines either the external image or the internal image included in the composite image as the specific image based on the state of the moving object detected in S32. Then, the image processing device 10 determines the region in the composite image where the specific image exists as the specific region.

[0110] In S34, the image processing device 10 compresses and encodes the composite image generated in S31 using a codec corresponding to the ROI, thereby generating a processed composite image in which the specific region determined in S33 within the composite image has higher image quality than other regions.

[0111] In S35, the image processing device 10 transmits the processed composite image to an external device. The external device performs predetermined processing on the received processed composite image. For example, the external device may perform any of the processes 1 to 3 described in the second embodiment, or it may perform other processes.

[0112] The image processing device 10 can repeat the above process. Furthermore, the image processing device 10 can perform the above process in real time.

[0113] Other configurations of the image processing device 10 can be the same as those in the first and second embodiments.

[0114] The image processing apparatus 10 of the third embodiment can achieve the same effects as the image processing apparatus 10 of the first and second embodiments. Furthermore, the image processing apparatus 10 can determine a specific region in the composite image that has higher image quality than other regions, depending on the state of the moving object.

[0115] For example, the image processing device 10 can detect whether the state of the moving object is in motion, stopped, immediately before stopping, immediately after stopping, immediately before starting, or immediately after starting, and determine a specific region corresponding to each state. With such an image processing device 10, it is possible to determine an appropriate specific region according to the motion state of the moving object.

[0116] Furthermore, the image processing device 10 can determine specific regions according to the internal state of the moving body. For example, if a passenger falls or the like occurs in the space where a passenger of the moving body is located, the image processing device 10 can determine the region in the internal image captured of that space (inside the moving body) as a specific region. Also, if an object falls or the like occurs in the space where an object is stored in the moving body (trunk room, cargo bed, etc.), the image processing device 10 can determine the region in the internal image captured of that space (inside the moving body) as a specific region. Also, if an abnormality occurs in the space where a component of the moving body is stored (engine room, etc.), the image processing device 10 can determine the region in the internal image captured of that space (inside the moving body) as a specific region.

[0117] With such an image processing device 10, it is possible to determine an appropriate specific region according to the internal state of the moving object.

[0118] Furthermore, the image processing device 10 can determine a specific region according to the external state of the moving object. For example, if a predetermined object is detected around the moving object, the image processing device 10 can determine the region in which the external image including that object is captured as a specific region.

[0119] With such an image processing device 10, it is possible to determine an appropriate specific region according to the external state of the moving object.

[0120] <<Modifications>> Below, modifications applicable to the first to third embodiments are described. In these modifications, the same effects and advantages as those of the first to third embodiments are achieved.

[0121] <Modification 1> In Modification 1, the processing unit 13 determines a specific image using the method described in the third embodiment, and then detects a predetermined object from within the specific image. The processing unit 13 then determines the "region in which the predetermined object exists within the region in which the specific image exists within the composite image" as the specific region.

[0122] The objects to be detected are predetermined. For example, certain types of objects such as people, other moving objects, animals, signs, and buildings may be designated as objects to be detected. Alternatively, all objects detected by the object detection process may be designated as targets for detection. Furthermore, the objects to be detected may be predetermined for each image generated by each camera. For example, in an image taken of a room where a person is located, the person may be designated as the target for detection. Also, in images taken of the front, rear, right side, left side, and near the door of a moving object, all objects detected by the object detection process may be designated as targets for detection.

[0123] The processing unit 13 detects a predetermined object within a specific image by analyzing the image. This image analysis is achieved using widely known techniques.

[0124] Figure 8 shows an example of a functional block diagram of this configuration. Images generated by multiple cameras are input to the image processing device 10. Although two cameras are shown in the figure, the number of cameras is not limited to these.

[0125] The image processing device 10 synthesizes the input images (image synthesis). Then, the image processing device 10 performs object detection processing on the synthesized image. Note that the image processing device 10 may also perform object detection processing on each of the multiple images before synthesis. The image processing device 10 also detects the state of the moving object. Then, the image processing device 10 determines a specific region based on the results of object detection and the state of the moving object. For example, a bandwidth limit is specified, and the image processing device 10 determines the specific region based on that.

[0126] Next, the image processing device 10 compresses and encodes the composite image using a codec (encoder) that supports ROI, thereby generating a processed composite image in which a specific region within the composite image has higher image quality than other regions. Then, the image processing device 10 transmits the processed composite image to an external device, for example, via TCP (Transmission Control Protocol) / IP (Internet Protocol) communication. Upon receiving the processed composite image, the external device decodes and reproduces it.

[0127] <Modification 2> In Modification 2, the synthesis unit 12 generates a composite image that includes a portion of the multiple images (external image and internal image) acquired by the acquisition unit 11. That is, the synthesis unit 12 selects a portion of the multiple images (external image and internal image) acquired by the acquisition unit 11, and generates a composite image by arranging the selected images in a predetermined layout, for example, as shown in Figure 5.

[0128] Here, we will explain an example of the process of selecting a portion of multiple images. In this example, it is pre-configured to select images generated by a predetermined camera among multiple cameras. This setting can be changed by user input. For example, a driver riding in the mobile vehicle, a monitor who monitors the status of the mobile vehicle based on the images displayed on the processing center's display as described in the first embodiment, or a remote operator who remotely controls the mobile vehicle based on those images can make this user input. The user can also configure the system to select images generated by other cameras. Furthermore, the user may change the number of images to select, i.e., the number of images to include in the composite image, in this configuration. The processing unit 13 determines the images to select, i.e., the images to include in the composite image, according to the settings when processing the composite image.

[0129] In other examples, the processing unit 13 determines the images to select, i.e., the images to include in the composite image, according to the state of the moving object. The processing unit 13 may also determine the number of images to select, i.e., the number of images to include in the composite image, according to the state of the moving object. The state of the moving object is as described in the third embodiment.

[0130] The image processing device 10 has pre-registered rules that specify the images to be selected and the number of images to be selected for each state of the moving object. When the processing unit 13 identifies the state of the moving object, it refers to these rules and determines the images to be selected and the number of images to be selected for the identified state of the moving object. Then, the processing unit 13 arranges the identified images in a predetermined layout, for example as shown in Figure 5, to generate a composite image.

[0131] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0132] Furthermore, the flowcharts used in the above description show multiple steps (processes) in sequence. However, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content.

[0133] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. An image processing apparatus comprising: acquisition means for acquiring an external image of the outside of a moving body and an internal image of the inside of a moving body; synthesis means for generating a composite image by combining the external image and the internal image; and processing means for generating a processed composite image in which a specific region within the composite image has higher image quality than other regions. 2. The image processing apparatus according to 1, wherein the processing means detects the state of the moving body and changes the region in the composite image that is designated as the specific region according to the state of the moving body. 3. The image processing apparatus according to 2, wherein the processing means determines one of the external image and the internal image as the specific image based on the state of the moving body, and determines the region in the composite image in which the specific image exists as the specific region. 4. The image processing apparatus according to 3, wherein the processing means detects a predetermined object from the specific image and determines the region in the region in which the predetermined object exists within the region in which the specific image exists as the specific region. 5. The image processing apparatus according to 3 or 4, wherein the processing means detects at least one of the following states of the moving body: while moving, stopped, immediately before stopping, immediately after stopping, immediately before starting, and immediately after starting. 6. The image processing apparatus according to any one of 3 to 5, wherein the acquisition means acquires a forward image taken of the front of the moving body as the external image, the processing means detects that the moving body is moving, and if the moving body is moving, determines the forward image as the specific image.7. The image processing apparatus according to any one of 3 to 5, wherein the acquisition means acquires a front image taken of the front of the moving body, a right image taken of the right side of the moving body, a left image taken of the left side of the moving body, and a rear image taken of the rear of the moving body as the external images, the processing means detects that the moving body is in motion as the state of the moving body, further detects at least one of the speed, acceleration, steering wheel state, and whether the moving body is moving forward or backward as the state of the moving body, and determines a part of the front image, the right image, the left image, and the rear image as the specific image based on at least one of the speed, acceleration, steering wheel state, and whether the moving body is moving forward or backward. 8. The image processing apparatus according to any one of 3 to 7, wherein the acquisition means acquires a door-area image taken of the area near the door of the moving body as the external image, the processing means detects that the moving body has just stopped as the state of the moving body, and if the moving body has just stopped, determines the door-area image as the specific image. 9. The image processing apparatus according to any one of 3 to 8, wherein the acquisition means acquires a front image taken of the front of the moving body and a rear image taken of the rear of the moving body as the external image, and the processing means detects that the moving body is about to depart as the state of the moving body, and if the moving body is about to depart, the front image, the rear image, and the internal image in the composite image are determined to be the specific image. 10. The image processing apparatus according to any one of 3 to 9, wherein the processing means detects the occurrence of a predetermined event inside the moving body as the state of the moving body, and if the occurrence of the predetermined event inside the moving body is detected, the internal image is determined to be the specific image. 11. The image processing apparatus according to any one of 2 to 10, wherein the processing means detects the state of the moving body based on at least one of the position of the moving body, the external image, the internal image, and information collected by a sensor mounted on the moving body. 12. The image processing apparatus according to 11, wherein the processing means detects that the state of the moving body is about to depart based on the open / closed state of the doors of the moving body.13. The image processing apparatus according to 12, wherein the processing means determines that the state of the mobile body is immediately before departure when the open / closed state of the door of the mobile body changes from an open state to a closed state. 14. The image processing apparatus according to any one of 11 to 13, wherein the processing means detects that the state of the mobile body is immediately before stopping based on the distance between the position of the mobile body and the next stopping position. 15. The image processing apparatus according to 14, wherein the processing means determines that the state of the mobile body is immediately before stopping when the distance is less than or equal to a threshold. 16. The image processing apparatus according to any one of 1 to 15, wherein the processing means compresses and encodes the composite image to generate the processed composite image. 17. The image processing apparatus according to any one of 1 to 16, further comprising a transmission means for transmitting the processed composite image to an external device. 18. The image processing apparatus according to any one of 1 to 17, mounted on the mobile body. 19. Image processing method comprising: one or more computers acquiring external images of the outside of a moving object and internal images of the inside of a moving object; generating a composite image by combining the external image and the internal image; and generating a processed composite image in which a specific region within the composite image has higher image quality than other regions. 20. A recording medium for recording a program that causes a computer to function as: acquisition means for acquiring external images of the outside of a moving object and internal images of the inside of a moving object; synthesis means for generating a composite image by combining the external image and the internal image; and processing means for generating a processed composite image in which a specific region within the composite image has higher image quality than other regions.

[0134] Some or all of the components of components 2 through 18 that are dependent on the image processing apparatus described in component 1 above may also be dependent on the image processing method described in component 19 and the recording medium described in component 20, in the same manner as the dependency relationship between components 1 and components 2 through 18. Furthermore, without departing from the embodiments described above, some or all of the configurations described in the components can be realized in various hardware, software, various recording means for recording software, or systems.

[0135] 10 Image processing device 11 Acquisition unit 12 Synthesis unit 13 Processing unit 14 Transmission unit 1A Processor 2A Memory 3A Input / Output I / F 4A Peripheral circuitry 5A Bus

Claims

1. An image processing apparatus comprising: acquisition means for acquiring an external image taken of the outside of a moving object and an internal image taken of the inside of a moving object; synthesis means for generating a composite image by combining the external image and the internal image; and processing means for generating a processed composite image in which a specific region within the composite image has higher image quality than other regions.

2. The image processing apparatus according to claim 1, wherein the processing means detects the state of the moving object and changes the region in the composite image that is designated as the specific region according to the state of the moving object.

3. The image processing apparatus according to claim 2, wherein the processing means determines, based on the state of the moving body, either the external image or the internal image as a specific image, and determines the region in the composite image in which the specific image exists as the specific region.

4. The image processing apparatus according to claim 3, wherein the processing means detects a predetermined object from the specific image and determines the region in the composite image where the specific image exists where the predetermined object exists as the specific region.

5. The image processing apparatus according to claim 3 or 4, wherein the processing means detects at least one of the following states of the moving body: while moving, while stopped, immediately before stopping, immediately after stopping, immediately before starting, and immediately after starting.

6. The image processing apparatus according to any one of claims 3 to 5, wherein the acquisition means acquires a forward image taken of the front of the moving body as the external image, and the processing means detects that the moving body is in motion as the state of the moving body, and if the moving body is in motion, determines the forward image as the specific image.

7. The image processing apparatus according to any one of claims 3 to 5, wherein the acquisition means acquires a front image taken of the front of the moving body, a right image taken of the right side of the moving body, a left image taken of the left side of the moving body, and a rear image taken of the rear of the moving body as the external images; the processing means detects that the moving body is in motion as the state of the moving body; further detects at least one of the speed, acceleration, steering wheel state, and whether the moving body is moving forward or backward as the state of the moving body; and determines a portion of the front image, the right image, the left image, and the rear image as the specific image based on at least one of the speed, acceleration, steering wheel state, and whether the moving body is moving forward or backward.

8. The image processing apparatus according to any one of claims 3 to 7, wherein the acquisition means acquires an image of the area near the door of the moving body as the external image, and the processing means detects that the moving body has just stopped as the state of the moving body, and if the moving body has just stopped, determines the image of the area near the door as the specific image.

9. The image processing apparatus according to any one of claims 3 to 8, wherein the acquisition means acquires a front image taken of the front of the moving body and a rear image taken of the rear of the moving body as the external image, and the processing means detects that the state of the moving body is that the moving body is about to depart, and if the moving body is about to depart, the front image, the rear image, and the internal image in the composite image are determined to be the specific image.

10. The image processing apparatus according to any one of claims 3 to 9, wherein the processing means detects the occurrence of a predetermined event inside the moving body as the state of the moving body, and determines the internal image as the specific image when the occurrence of the predetermined event inside the moving body is detected.

11. The image processing apparatus according to any one of claims 2 to 10, wherein the processing means detects the state of the moving body based on at least one of the position of the moving body, the external image, the internal image, and information collected by a sensor mounted on the moving body.

12. The image processing apparatus according to claim 11, wherein the processing means detects that the state of the moving body is immediately before departure based on the open / closed state of the door of the moving body.

13. The image processing apparatus according to claim 12, wherein the processing means determines that the state of the moving body is immediately before departure when the open / closed state of the door of the moving body changes from an open state to a closed state.

14. The image processing apparatus according to any one of claims 11 to 13, wherein the processing means detects that the state of the moving body is immediately before stopping, based on the distance between the position of the moving body and the next stopping position.

15. The image processing apparatus according to claim 14, wherein the processing means determines that the state of the moving object is immediately before stopping when the distance is less than or equal to a threshold.

16. The image processing apparatus according to any one of claims 1 to 15, wherein the processing means compresses and encodes the composite image to generate the processed composite image.

17. The image processing apparatus according to any one of claims 1 to 16, further comprising a transmission means for transmitting the processed composite image to an external device.

18. An image processing apparatus according to any one of claims 1 to 17, mounted on the mobile body.

19. An image processing method comprising: one or more computers acquiring external images of the outside of a moving object and internal images of the inside of a moving object; generating a composite image by combining the external image and the internal image; and generating a processed composite image in which a specific region within the composite image has higher image quality than other regions.

20. A recording medium for recording a program that causes a computer to function as: an acquisition means for acquiring an external image taken of the outside of a moving object and an internal image taken of the inside of a moving object; a synthesis means for generating a composite image by combining the external image and the internal image; and a processing means for generating a processed composite image in which a specific region within the composite image has higher image quality than other regions.

Images