Control devices and methods for mobile bodies, storage media, and vehicles

Abstract

This invention relates to a control device and control method for a mobile body, a storage medium, and a vehicle, which appropriately identifies the external environment of the mobile body. A control device for a mobile body having multiple imaging devices, including a first imaging device and a second imaging device, comprises: an image acquisition unit that acquires images of the external environment of the mobile body from the multiple imaging devices; a correction unit that performs distortion reduction processing on one or more regions contained in the images acquired from the multiple imaging devices to reduce image distortion; and a recognition unit that recognizes the external environment of the mobile body based on the distortion-reduced images. The correction unit, when selecting a first region in the image acquired from the first imaging device that contains a state of a specific orientation of the mobile body as the object of distortion reduction processing, also selects a second region in the image acquired from the second imaging device that contains a state of a specific orientation as the object of distortion reduction processing.

Description

Technical Field

[0001] This invention relates to a control device and control method for a mobile body, a storage medium, and a vehicle. Background Technology

[0002] The technology of using multiple cameras to identify the vehicle's surroundings has been put into practical use. The results of the external environment identification are used in driver assistance and autonomous driving. Patent Document 1 proposes a technology that uses a wide-angle lens camera to capture a wide range of images of the vehicle's surroundings. Coordinate transformation is performed to reduce distortion in the images captured by the wide-angle lens camera.

[0003] Existing technical documents

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-171964 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] Processing to reduce distortion in images captured by cameras equipped with wide-angle or fisheye lenses consumes power. Therefore, excessive distortion reduction processing in order to identify the outside world of a vehicle increases power consumption. This increase in power consumption is not limited to vehicles but also applies to other moving objects. Part of the object of this invention is to provide a technique for properly identifying the outside world of a moving object.

[0008] means for solving problems

[0009] In view of the above problems, according to a partial embodiment, a control device is provided, which is a control device for a moving body having a plurality of shooting devices, including a first shooting device and a second shooting device. The control device includes: an image acquisition unit that acquires images of the outside world of the moving body from the plurality of shooting devices; a correction unit that performs distortion reduction processing on one or more regions contained in the images acquired from the plurality of shooting devices to reduce image distortion; and an identification unit that identifies the outside world of the moving body based on the images after distortion reduction processing. The correction unit selects a first region in the image acquired from the first shooting device that contains a state of a specific direction of the moving body as the object of the distortion reduction processing, and selects a second region in the image acquired from the second shooting device that contains a state of the specific direction as the object of the distortion reduction processing. According to another embodiment, a control method is provided for a moving body having multiple shooting devices, the multiple shooting devices including a first shooting device and a second shooting device. The control method includes: an image acquisition step, in which images of the outside world of the moving body are acquired from the one or more shooting devices; a correction step, in which distortion reduction processing is performed on one or more regions contained in the images acquired from the one or more shooting devices to reduce image distortion; and an identification step, in which the outside world of the moving body is identified based on the images after distortion reduction processing, and in the correction step, when a first region in the image acquired from the first shooting device that contains a state of a specific direction of the moving body is selected as the object of the distortion reduction processing, a second region in the image acquired from the second shooting device that contains a state of the specific direction is selected as the object of the distortion reduction processing.

[0010] Invention Effects

[0011] Based on the above methods, the external environment of a moving object can be appropriately identified. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating an example of the vehicle configuration involved in the implementation method.

[0013] Figure 2 This is a schematic diagram illustrating the shooting range of the camera involved in the implementation method.

[0014] Figure 3 This is a schematic diagram illustrating the distortion reduction process involved in the implementation method.

[0015] Figure 4 This is a schematic diagram illustrating the object area involved in the distortion reduction processing according to the implementation method.

[0016] Figure 5 This is a flowchart illustrating an example of the operation of the vehicle control device according to the embodiment.

[0017] Figure 6 This is a timing diagram illustrating the candidate objects for distortion reduction processing involved in the implementation method.

[0018] Figure 7 This is a schematic diagram illustrating a typical driving scenario involved in the implementation method.

[0019] Figure 8 This is a schematic diagram illustrating the state transitions according to specific rules involved in the implementation method.

[0020] Figure 9 This is a schematic diagram illustrating the vertical position of a region according to a specific rule involved in the implementation method.

[0021] Figure 10 This is a schematic diagram illustrating a specific driving scenario involved in the implementation method.

[0022] Figure 11 This is a schematic diagram illustrating the state transitions according to specific rules involved in the implementation method.

[0023] Figure 12 This is a schematic diagram illustrating the state transitions according to specific rules involved in the implementation method.

[0024] Figure 13 This is a schematic diagram illustrating the vertical position of a region according to a specific rule involved in the implementation method. Detailed Implementation

[0025] The embodiments will now be described in detail with reference to the accompanying drawings. It should be noted that the following embodiments are not intended to limit the invention, and that not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more features from the plurality of features described in the embodiments may be arbitrarily combined. Furthermore, identical or identical components are labeled with the same reference numerals, and repeated descriptions are omitted. It should be noted that in the following embodiments, a vehicle is used as the mobile body for description; however, the mobile body is not limited to a vehicle but may also be a flying object, a robot, etc.

[0026] Figure 1 This is a block diagram of a vehicle 1 according to one embodiment of the present invention. Figure 1 The diagram shows a top view and a side view of vehicle 1. As an example, vehicle 1 is a four-wheeled passenger car of sedan type. Vehicle 1 can be a four-wheeled vehicle as described above, or it can be a two-wheeled vehicle or other types of vehicles.

[0027] Vehicle 1 includes a vehicle control device 2 (hereinafter referred to as control device 2) for controlling vehicle 1. Control device 2 includes multiple ECUs (Electronic Control Units) 20 to ECU 29 that are connected and able to communicate via an in-vehicle network. Each ECU includes a processor such as a CPU (Central Processing Unit), a memory such as semiconductor memory, and an interface for external devices. The memory stores the program executed by the processor, the data used by the processor in processing, etc. Each ECU may have multiple processors, memories, and interfaces. For example, ECU 20 has a processor 20a and a memory 20b. Processing based on ECU 20 is performed by the processor 20a executing instructions, including the program stored in memory 20b. Instead, ECU 20 may have a dedicated integrated circuit such as an ASIC (Application Specific Integrated Circuit) for performing processing based on ECU 20. The same applies to other ECUs.

[0028] The functions of each ECU 20 to 29 will be explained below. It should be noted that the number of ECUs and their functions can be designed appropriately, and can be more detailed or integrated than in this embodiment.

[0029] ECU 20 performs controls related to autonomous driving of vehicle 1. In autonomous driving, it automatically controls at least one of the steering, acceleration, and deceleration of vehicle 1. The autonomous driving performed by ECU 20 may include autonomous driving that does not require driving operations by the driver (also known as autonomous driving) and autonomous driving that assists the driver in driving operations (also known as driving assistance).

[0030] ECU 21 controls the electric power steering system 3. The electric power steering system 3 includes a mechanism for steering the front wheels according to the driver's driving operation (steering operation) on the steering wheel 31. Additionally, the electric power steering system 3 includes a motor that provides driving force for assisting steering operation or automatically steering the front wheels, a sensor for detecting the steering angle, etc. When the vehicle 1 is in automatic driving mode, ECU 21 automatically controls the electric power steering system 3 in accordance with instructions from ECU 20 to control the direction of travel of the vehicle 1.

[0031] ECU 22 and ECU 23 control the detection unit that detects the surrounding conditions of the vehicle and process the detection results. As the detection unit for detecting the vehicle's condition, vehicle 1 includes one standard camera 40 and four fisheye cameras 41-44. Standard camera 40 and fisheye cameras 42 and 44 are connected to ECU 22. Fisheye cameras 41 and 43 are connected to ECU 23. ECU 22 and 23 can extract the outlines of objects and lane markings (white lines, etc.) on the road by analyzing the images captured by standard camera 40 and fisheye cameras 41-44.

[0032] Fisheye cameras 41-44 refer to cameras equipped with fisheye lenses. The configuration of fisheye camera 41 will be described below. Other fisheye cameras 42-44 may also have the same configuration. The field of view of fisheye camera 41 is wider than that of standard camera 40. Therefore, fisheye camera 41 can capture a wider range than standard camera 40. The image captured by fisheye camera 41 has greater distortion compared to the image captured by standard camera 40. Therefore, ECU 23 can perform distortion reduction processing (hereinafter referred to as "distortion reduction processing") on the image before analyzing it. On the other hand, ECU 22 can skip distortion reduction processing on the image captured by standard camera 40 before analyzing it. Thus, standard camera 40 is an imaging device that captures images that are not subject to distortion reduction processing, while fisheye camera 41 is an imaging device that captures images that are subject to distortion reduction processing. Instead of the standard camera 40, other shooting devices that can be used to capture images of objects that do not undergo distortion reduction processing, such as cameras equipped with wide-angle lenses or telephoto lenses, can be used.

[0033] A standard camera 40 is mounted at the center of the front of vehicle 1 to photograph the situation in front of vehicle 1. A fisheye camera 41 is mounted at the center of the front of vehicle 1 to photograph the situation in front of vehicle 1. Figure 1 In this design, the standard camera 40 and the fisheye camera 41 are shown arranged horizontally. However, the configuration of the standard camera 40 and the fisheye camera 41 is not limited to this; for example, they can also be arranged vertically. Furthermore, at least one of the standard camera 40 and the fisheye camera 41 can be mounted on the front of the roof of the vehicle 1 (e.g., inside the windshield). For example, the fisheye camera 41 can be mounted at the center of the front of the vehicle 1 (e.g., on the bumper), and the standard camera 40 can be mounted at the front of the roof of the vehicle 1. The fisheye camera 42 is mounted at the center of the right side of the vehicle 1 to capture images of the right side of the vehicle 1. The fisheye camera 43 is mounted at the center of the rear of the vehicle 1 to capture images of the rear of the vehicle 1. The fisheye camera 44 is mounted at the center of the left side of the vehicle 1 to capture images of the left side of the vehicle 1.

[0034] The types, number, and mounting positions of the cameras in vehicle 1 are not limited to the examples mentioned above. In addition, as a detection unit for detecting targets around vehicle 1 or measuring the distance to targets, vehicle 1 may also include optical radar (Light Detection and Ranging) or millimeter-wave radar.

[0035] ECU 22 controls the standard camera 40 and fisheye cameras 42 and 44, and processes the information from the detection results. ECU 23 controls the fisheye cameras 41 and 43, and processes the information from the detection results. By dividing the detection unit that detects the condition of the vehicle into two systems, the reliability of the detection results can be improved.

[0036] ECU 24 controls the gyroscope sensor 5, GPS sensor 24b, and communication device 24c, and processes the detection or communication results. The gyroscope sensor 5 detects the rotational motion of vehicle 1. It can determine the vehicle 1's route based on the detection results of the gyroscope sensor 5, wheel speed, etc. The GPS sensor 24b detects the current position of vehicle 1. The communication device 24c wirelessly communicates with a server providing map and traffic information to obtain this information. ECU 24 can access a map information database 24a built in memory to perform route searches from the current location to the destination, etc. ECU 24, map database 24a, and GPS sensor 24b constitute a navigation device.

[0037] The ECU 25 is equipped with a communication device 25a for vehicle-to-vehicle communication. The communication device 25a communicates wirelessly with other vehicles in the vicinity and exchanges information between vehicles.

[0038] ECU 26 controls the power unit 6. The power unit 6 is a mechanism that outputs driving force to rotate the drive wheels of vehicle 1, and includes, for example, an engine and a transmission. ECU 26 controls the engine output, for example, in response to driver operations (accelerator or acceleration) detected by the operation detection sensor 7a located on the accelerator pedal 7A, or switches transmission gears based on information such as vehicle speed detected by the vehicle speed sensor 7c. When vehicle 1 is in automatic driving mode, ECU 26 automatically controls the power unit 6 in response to instructions from ECU 20, controlling the acceleration and deceleration of vehicle 1.

[0039] ECU27 controls lighting devices (headlights, taillights, etc.), including the turn indicator 8 (turn signal). Figure 1 In the example, the direction indicator 8 is located at the front, door rearview mirror and rear of the vehicle 1.

[0040] ECU 28 controls the input / output device 9. The input / output device 9 outputs information to the driver and receives information from the driver. The sound output device 91 notifies the driver of information via sound. The display device 92 notifies the driver of information via image display. The display device 92 may be located in front of the driver's seat, forming part of the instrument panel, etc. It should be noted that sound and display are exemplified here, but information can also be notified via vibration or light. Furthermore, multiple combinations of sound, display, vibration, or light can be used to report information. Further, the combination or reporting method can be different depending on the level of the information to be reported (e.g., urgency). The input device 93 is a switch group configured in a position operable by the driver to provide instructions for the vehicle 1, but it may also include a sound input device.

[0041] ECU 29 controls the braking device 10 and the parking brake (not shown). The braking device 10 is, for example, a disc brake, installed on each wheel of the vehicle 1, which decelerates or stops the vehicle 1 by applying resistance to the rotation of the wheels. ECU 29 controls the operation of the braking device 10 in accordance with the driver's driving operation (braking operation) detected by the operation detection sensor 7b installed on the brake pedal 7B. When the vehicle 1 is in automatic driving mode, ECU 29 automatically controls the braking device 10 in accordance with the instructions from ECU 20, controlling the deceleration and stopping of the vehicle 1. The braking device 10 and the parking brake can also operate to maintain the vehicle 1 in a stopped state. In addition, if the transmission of the power unit 6 is equipped with a parking lock mechanism, it can also be operated to maintain the vehicle 1 in a stopped state.

[0042] Reference Figure 2 The shooting range of the standard camera 40 and the fisheye cameras 41-44 is explained. Figure 2 In the diagram, (a) represents the horizontal shooting range of each camera. Figure 2 (b) in the figure represents the vertical shooting range of the fisheye camera 41 at the front of vehicle 1. Figure 2 (c) in the figure represents the vertical shooting range of the fisheye camera 42 on the right side of vehicle 1. Figure 2 In this specification, (d) indicates the vertical shooting range of the fisheye camera 43 at the rear of vehicle 1. In this specification, the horizontal and vertical directions are based on the body of vehicle 1. The vertical shooting range of the fisheye camera 42 on the left side of vehicle 1 can be compared with... Figure 2 (c) is the same as in the original text, so it is omitted.

[0043] First, refer to Figure 2(a) describes the shooting range in the top view of vehicle 1 (i.e., the horizontal direction of vehicle 1). Standard camera 40 captures images of the conditions encompassed by the shooting range 200. The shooting center 200C of standard camera 40 faces the front of vehicle 1. The shooting center 200C can also be defined by the direction of the optical axis of the lens. The horizontal field of view of standard camera 40 can be less than 90°, for example, approximately 45° or 30°.

[0044] Fisheye camera 41 captures the scene within the shooting range 201. The shooting center 201C of fisheye camera 41 faces the front of vehicle 1. Fisheye camera 42 captures the scene within the shooting range 202. The shooting center 202C of fisheye camera 42 faces the right front of vehicle 1. Fisheye camera 43 captures the scene within the shooting range 203. The shooting center 203C of fisheye camera 43 faces the rear front of vehicle 1. Fisheye camera 44 captures the scene within the shooting range 204. The shooting center 204C of fisheye camera 44 faces the left front of vehicle 1. The horizontal field of view of fisheye cameras 41-44 can be greater than 90°, greater than 150°, or greater than 180°, for example, approximately 180°. Figure 2 (a) shows an example of a fisheye camera 41-44 with a horizontal field of view of 180°.

[0045] Next, refer to Figure 2 (b) in the middle Figure 2 (d) in the diagram describes the shooting range of vehicle 1 in the vertical direction. Figure 2 In section (b), the vertical shooting range of the fisheye camera 41 is explained. Figure 2 In section (c), the vertical shooting range of the fisheye camera 42 is explained. Figure 2 Section (d) describes the vertical shooting range of fisheye camera 43. The vertical shooting range of other fisheye cameras 44 can also be described in the same way. Figure 2 (c) is the same.

[0046] The vertical field of view of fisheye cameras 41-44 can be greater than 90°, 150°, or 180°, for example, around 180°. Figure 2 (b) in the middle Figure 2In the text, (d) indicates an example where the vertical field of view of fisheye cameras 41-44 is 180°. The shooting center 203C of fisheye camera 43 faces downwards (towards the ground) compared to a direction parallel to the ground. Alternatively, the shooting center 203C of fisheye camera 43 can face either a direction parallel to the ground or a direction higher than parallel to the ground (opposite to the ground). Furthermore, the shooting centers 201C-204C of fisheye cameras 41-44 can also face a single direction in the vertical direction.

[0047] The standard camera 40 and the fisheye cameras 41-44 have shooting ranges 200-204 as described above, thus the frontal view and the four oblique directions of the vehicle 1 are included in the shooting range of two independent cameras. Specifically, the frontal view of the vehicle 1 is included in both the shooting range 200 of the standard camera 40 and the shooting range 201 of the fisheye camera 41. The right oblique front view of the vehicle 1 is included in both the shooting range 201 of the fisheye camera 41 and the shooting range 202 of the fisheye camera 42. The same applies to the other three oblique directions of the vehicle 1.

[0048] Reference Figure 3 as well as Figure 4 The distortion reduction processing of images captured by fisheye cameras 41-44 is explained. Figure 3 This represents the images before and after distortion reduction processing. Figure 4 This indicates the region that will be the target of distortion reduction processing. Figure 4 (a) in the diagram represents a view of vehicle 1 from above. Figure 4 Image (b) shows a view of vehicle 1 from the rear. Image 300 is an image of the right side of vehicle 1 taken by fisheye camera 42. Figure 3 As shown, image 300 exhibits significant distortion, particularly in the peripheral regions.

[0049] The ECU22, connected to the fisheye camera 42, performs distortion reduction processing on the image 300. Specifically, as... Figure 3 As shown, ECU22 determines a point within image 300 as the conversion center 301. (As...) Figure 4 As shown, the conversion center 301 is located on the right side of the shooting range 202 when viewed from the fisheye camera 42 in the horizontal direction, and faces a direction parallel to the ground in the vertical direction.

[0050] ECU22 extracts a rectangular region 302 from image 300, centered at conversion center 301. For example... Figure 4As shown, region 302 corresponds to region 202R on the right side of the shooting range 202 as observed from the fisheye camera 42. ECU 22 generates a distortion-reduced image 303 by performing distortion reduction processing on region 302. This image 303 is an image representing the condition of region 202R.

[0051] As a result of distortion reduction processing, the closer to the conversion center 301, the more distortion is reduced; conversely, in areas far from the conversion center 301, distortion is not reduced or even increases. When the entire image 300 is considered as the object of distortion reduction processing, distortion increases in regions located far from the conversion center 301. Therefore, even when using regions located at these distant locations to resolve the external environment of the vehicle 1, high-precision resolution is not possible. Therefore, the control device 2 sets the conversion center 301 as the region to be resolved, performs distortion reduction processing on the region surrounding the conversion center 301, and uses the processed image to resolve the condition of the region to be resolved.

[0052] Shooting range 201 includes the area 201L shooting the left diagonally front of vehicle 1, the area 201F shooting the front of vehicle 1 directly, and the area 201R shooting the right diagonally front of vehicle 1 as the area to be analyzed. Shooting range 202 includes the area 202L shooting the right diagonally front of vehicle 1, the area 202F shooting the right front of vehicle 1, and the area 202R shooting the right diagonally rear of vehicle 1 as the area to be analyzed. Shooting range 203 includes the area 203L shooting the right diagonally rear of vehicle 1, the area 203F shooting the rear front of vehicle 1, and the area 203R shooting the left diagonally rear of vehicle 1 as the area to be analyzed. Shooting range 204 includes the area 204L shooting the left diagonally rear of vehicle 1, the area 204F shooting the left front of vehicle 1, and the area 204R shooting the left diagonally front of vehicle 1 as the area to be analyzed. The shooting range 201 can also be divided into three equal regions 201L, 201F, and 201R in the horizontal direction (i.e., with equal horizontal field of view for each region). The other shooting ranges 202 to 204 can also be divided into three equal parts.

[0053] When the control device 2 wants to analyze the situation at the right front of the vehicle 1, it sets a conversion center 301 within the area 202L (e.g., the center of area 202L) encompassed by the shooting range 202 of the fisheye camera 42, performs distortion reduction processing on the area surrounding the conversion center 301, and uses the processed image to analyze the situation at the right front. When the control device 2 wants to analyze the situation at the right front of the vehicle 1, it sets a conversion center 301 within the area 202F (e.g., the center of area 202F) encompassed by the shooting range 202 of the fisheye camera 42, performs distortion reduction processing on the area surrounding the conversion center 301, and uses the processed image to analyze the situation at the right front. When the control device 2 wants to analyze the situation at the right rear of the vehicle 1, it sets a conversion center 301 within the area 202R (e.g., the center of area 202R) encompassed by the shooting range 202 of the fisheye camera 42, performs distortion reduction processing on the area surrounding the conversion center 301, and uses the processed image to analyze the situation at the right rear.

[0054] Reference Figure 5 An example of a method for controlling vehicle 1 by control device 2 in one embodiment will be described. This method can also be performed by the processor 20a of each ECU 20 to 29 of control device 2 executing a program stored in memory 20b. Figure 5 The method can also be started when the driving assistance function or automatic driving function of the control device 2 is turned on.

[0055] In step S501, the control device 2 acquires images of the external environment of the vehicle 1 from the standard camera 40 and the fisheye cameras 41-44, respectively. Each image includes images of the external environment of the vehicle 1. Figure 2 The situation described in the text refers to the scope of the situation.

[0056] In step S502, the control device 2 determines the current driving scenario of the vehicle 1. In the example described below, the scenario of driving on a narrow road is processed as the driving scenario of the vehicle. Other scenarios are processed as normal (default) scenarios. Normal scenarios include, for example, the scenario of the vehicle 1 driving along a road.

[0057] In step S503, the control device 2 determines one or more regions in the image acquired in step S501 that are objects to be processed for distortion reduction, according to rules corresponding to the current driving scenario of the vehicle 1. Hereinafter, this rule will be referred to as the region determination rule. The region determination rule is predetermined, for example, stored in memory 20b. Specific examples of the region determination rule will be described later.

[0058] In step S504, as Figure 3As shown, the control device 2 performs distortion reduction processing on one or more regions of the object determined to be subject to distortion reduction processing. This distortion reduction processing is used to reduce distortion in the images acquired from fisheye cameras 41-44. Existing techniques can also be used in the distortion reduction processing, so detailed descriptions are omitted. Distortion reduction processing is not required for the images acquired from the standard camera 40.

[0059] In step S505, the control device 2 identifies the external environment of the vehicle 1 based on images acquired from the standard camera 40 and images acquired from fisheye cameras 41-44 after distortion reduction processing. For example, the control device 2 can also determine objects around the vehicle 1 by applying the corrected images to a model learned in advance and stored in memory 20b. Furthermore, the control device 2 can also control the vehicle 1 based on the identification results of the external environment (e.g., automatic braking, notification to the driver, change of autonomous driving level, etc.). Existing technologies can also be applied to the control of the vehicle 1 in response to the identification results of the external environment, so detailed descriptions are omitted.

[0060] In step S506, the control device 2 determines whether to end the action. If the control device 2 determines that the action should be ended ("Yes" in step S506), it ends the action; otherwise ("No" in step S506), it returns the action to step S501. For example, the control device 2 may also determine whether to end the action based on whether the driving assistance function or the automatic driving function is turned off.

[0061] As described above, steps S501 to S505 are repeated. The control device 2 may also periodically execute the actions of steps S501 to S505. The execution cycle varies depending on the time required for the distortion reduction processing in S504 and the recognition processing in S505, and may be approximately 100 ms.

[0062] Reference Figure 6 The periodic operation of control device 2 will be explained. Figure 6 The circles represent candidate images used in the recognition process of step S505. Figure 6Δt represents the period of execution steps S501 to S505. Since the image acquired from the standard camera 40 has low distortion, it can be used in the recognition process of step S505 without performing the distortion reduction processing of step S504. The images acquired from the fisheye cameras 41 to 44 are used in the recognition process of step S505 after undergoing distortion reduction processing in step S504. As described above, the image acquired from the fisheye camera can generate an image in which distortion reduction processing has been performed on three regions divided in the horizontal direction. Therefore, in one operation of a periodic operation, the control device 2 can use up to thirteen images to perform the recognition process. Since twelve of the thirteen images are acquired from the fisheye camera, distortion reduction processing is performed before the recognition process. If distortion reduction processing is performed on all twelve regions, the processing load increases, and the power consumption also increases. Therefore, in the following embodiment, based on the driving scenario of the vehicle 1, it is determined which of the twelve regions will undergo distortion reduction processing and be used for recognition processing at each operation.

[0063] Reference Figures 7 to 9 This section explains the rules for determining the area in typical scenarios. Figure 7 This illustrates an example of a driving scenario. Figure 7 In the example shown, vehicle 1 is traveling along a straight road.

[0064] Figure 8 This indicates the region that becomes the object of analysis at each action timing. Control device 2 sequentially repeats states 800 to 802. That is, if the action timing at time t is state 800, then the action timing at time t+Δt (as mentioned above, Δt represents the period) becomes state 801, the action timing at time t+2×Δt becomes state 802, and the action timing at time t+3×Δt returns to state 800. The same rules apply to the region determination rules for other driving scenarios described later.

[0065] In state 800, the shooting range 200 of the standard camera 40, area 201L of the fisheye camera 41, area 202R of the fisheye camera 42, area 203L of the fisheye camera 43, and area 204R of the fisheye camera 44 become the objects of analysis. In state 801, the shooting range 200 of the standard camera 40, area 201F of the fisheye camera 41, area 202F of the fisheye camera 42, area 203F of the fisheye camera 43, and area 204F of the fisheye camera 44 become the objects of analysis. In state 802, the shooting range 200 of the standard camera 40, area 201R of the fisheye camera 41, area 202L of the fisheye camera 42, area 203R of the fisheye camera 43, and area 204L of the fisheye camera 44 become the objects of analysis.

[0066] When the area captured by the fisheye camera becomes the object of resolution, the control device 2 performs distortion reduction processing on that area as described above. Therefore, the area determination rule specifies the horizontal position of the area that becomes the object of distortion reduction processing, and the timing when the area located at that position is the object of distortion reduction processing. In addition, the area determination rule specifies rules separately for each of the plurality of fisheye cameras 41 to 44.

[0067] As described above, by changing states at each operational moment, the control device 2 sets the frontal view of the vehicle 1 as the analysis object for each cycle (i.e., each time), and sets the right-side front, right-side front, right-side rear, rear front, left-side rear, left-side front, and left-side front as analysis objects for each of three cycles. Furthermore, to prevent the load from being concentrated on the control device 2 at specific operational moments, areas not considered as analysis objects are distributed across multiple operational moments. Moreover, for the frontal view of the vehicle 1, analysis is performed every three cycles using both the image from the standard camera 40 and the image from the fisheye camera; for each of the four tilt directions of the vehicle 1, analysis is performed every three cycles using both the image from two fisheye cameras. In this way, by using a portion of the images from the fisheye cameras 41-44, which undergo distortion correction processing, as the analysis object at each operational moment, the processing load on the control device 2 is reduced, and power consumption is decreased.

[0068] Figure 9 This indicates the vertical position of the region of the parsed object. Figure 9 In the figure, (a) represents the vertical position of region 201F of the fisheye camera 41. Figure 9 (b) in the figure represents the vertical position of region 202F of the fisheye camera 42. Figure 9 In the diagram, (c) represents the vertical position of region 203F of fisheye camera 43. The vertical positions of each region of fisheye camera 44 can also be compared with... Figure 9 The same as (b) in the previous example, so the explanation is omitted.

[0069] In the rules for determining the region, such as Figure 9As shown in (a), the angle between the conversion center 301 of region 201F and the vertical direction of vehicle 1 is defined as θ1. θ1 can be 90 degrees or a value less than 90 degrees (e.g., 80 degrees). For regions 201R and 201L, the angle between the conversion center 301 and the vertical direction of vehicle 1 can also be defined as θ1. Similarly, for the regions of fisheye cameras 42-44, the angle between the conversion center 301 and the vertical direction of vehicle 1 can also be defined as θ1. In this way, the region determination rule defines the vertical position of the region to which distortion reduction processing is applied. By setting the angle between the conversion center 301 and the vertical direction of vehicle 1 to θ1 (e.g., 90 degrees), it is possible to evenly resolve the distance and proximity of vehicle 1.

[0070] Reference Figures 10-13 The rules for determining the area in a scenario where vehicle 1 is driving on a narrow road are explained. A narrow road can also be a driving scenario where the distance between vehicle 1 and an obstacle is below a threshold (e.g., below 50cm). Figure 10 This is an example of such a scenario. Figure 10 (a) in the image represents the scene where vehicle 1 is driving on an S-shaped curve. Figure 10 (b) in the diagram represents the scenario where vehicle 1 is traveling on an L-shaped road. Figure 10 In vehicle 10, (c) represents a scenario where vehicle 1 meets an oncoming vehicle. In vehicle 10, (d) represents a scenario where vehicle 1 passes beside a vehicle that is about to turn right. Figure 10 (a) and Figure 10 As shown in (b), narrow roads are sometimes created based on the shape of the road, such as... Figure 10 (c) and Figure 10 As shown in (d) in the diagram, narrow roads sometimes arise depending on traffic conditions.

[0071] Figure 11This represents an example of a region-determining rule, where the region becomes the object of analysis at each action timing. Control device 2 sequentially repeats states 1100 to 1106. In state 1100, the shooting range 200 of standard camera 40, region 201F of fisheye camera 41, regions 202L and 202F of fisheye camera 42, and region 204R of fisheye camera 44 become the objects of analysis. In state 1101, the shooting range 200 of standard camera 40, region 201F of fisheye camera 41, regions 202L and 202R of fisheye camera 42, and region 204R of fisheye camera 44 become the objects of analysis. In state 1102, the shooting range 200 of standard camera 40, region 201F of fisheye camera 41, region 202L of fisheye camera 42, region 203L of fisheye camera 43, and region 204R of fisheye camera 44 become the objects of analysis. In state 1103, the shooting range 200 of the standard camera 40, area 201F of the fisheye camera 41, area 202L of the fisheye camera 42, area 203F of the fisheye camera 43, and area 204R of the fisheye camera 44 become the objects of analysis. In state 1104, the shooting range 200 of the standard camera 40, area 201F of the fisheye camera 41, area 202L of the fisheye camera 42, area 203R of the fisheye camera 43, and area 204R of the fisheye camera 44 become the objects of analysis. In state 1105, the shooting range 200 of the standard camera 40, area 201F of the fisheye camera 41, area 202L of the fisheye camera 42, and areas 204R and 204L of the fisheye camera 44 become the objects of analysis. In state 1106, the shooting range 200 of the standard camera 40, the area 201F of the fisheye camera 41, the area 202L of the fisheye camera 42, and the areas 204R and 204F of the fisheye camera 44 become the objects of analysis.

[0072] By changing states at each operational moment as described above, control device 2 analyzes the front, right-front, and left-front of vehicle 1 in one cycle (i.e., each time), the right front, rear front, and left front of vehicle 1 in seven cycles, and the right rear and left rear of vehicle 1 in two out of seven cycles. Furthermore, to prevent the load from concentrating on control device 2 at specific operational moments, areas not analyzed are distributed across multiple operational moments.

[0073] Figure 12This represents an example of a region-determining rule, where the region becomes the object of analysis at each action timing. Control device 2 sequentially repeats states 1200 to 1202. In state 1200, the shooting range 200 of standard camera 40, region 201F of fisheye camera 41, region 202L of fisheye camera 42, region 203F of fisheye camera 43, and region 204R of fisheye camera 44 become the objects of analysis. In state 1201, the shooting range 200 of standard camera 40, regions 202L and 202F of fisheye camera 42, and regions 204F and 204R of fisheye camera 44 become the objects of analysis. In state 1202, the shooting range 200 of standard camera 40, region 202L of fisheye camera 42, regions 203L and 203R of fisheye camera 43, and region 204R of fisheye camera 44 become the objects of analysis.

[0074] By changing states at each action timing as described above, control device 2 sets the front, right-front, and left-front of vehicle 1 as the analysis object for each cycle (i.e., each time), and sets the right front, right rear, rear front, left rear, and left front of vehicle 1 as the analysis object for each three cycles. Furthermore, to prevent the load from concentrating on control device 2 at specific action timings, areas not considered as analysis objects are distributed across multiple action timings.

[0075] Figure 13 This indicates the vertical position of the region of the parsed object. Figure 13 In the figure, (a) represents the vertical position of region 201F of the fisheye camera 41. Figure 13 (b) in the figure represents the vertical position of region 202F of the fisheye camera 42. Figure 13 In the diagram, (c) represents the vertical position of region 203F of fisheye camera 43. The vertical positions of each region of fisheye camera 44 can also be compared with... Figure 13 The same as (b) in the previous example, so the explanation is omitted.

[0076] In the rules for determining the region, such as Figure 13 As shown in (a), the angle formed by the conversion center 301 of the specified region 201F and the vertical downward direction of vehicle 1 is defined as θ2. θ2 is a ratio of Figure 9 The angle θ1 is a smaller value than θ2, for example, it can be 70 degrees. For regions 201R and 201L, the angle between the conversion center 301 and the vertical downward direction of vehicle 1 can also be defined as θ2. For the region of fisheye camera 42-44, the angle between the conversion center 301 and the vertical downward direction of vehicle 1 is also defined as θ3. θ3 is a smaller value than θ2, for example, it can be 45 degrees.

[0077] Thus, in any of the following regions of vehicle 1: frontal, right-slanted front, right-front, right-slanted rear, rear-front, left-slanted rear, left-front, and left-slanted front, the vertical position of the region to be analyzed when vehicle 1 is traveling on a narrow road is lower than when vehicle 1 is traveling outside of a narrow road (e.g., the usual scenario described above) (e.g., conversion center 301 is facing downwards). When vehicle 1 is traveling on a narrow road, its wheels may hit the curb or fall into a ditch. By placing the region to be analyzed lower, the accuracy of analyzing conditions near the ground is improved. Furthermore, in the region determination rule when vehicle 1 is traveling on a narrow road, the region 201F in front of the fisheye camera 41 becomes the region to be analyzed. Thus, as... Figure 13 As shown in (a), the area near the front of the vehicle 1, which is not included in the shooting range 200 of the standard camera 40, can be analyzed. Furthermore, the area to the right front of the vehicle 1 is analyzed based on the region 202L of the fisheye camera 42. Thus, the area near the front wheel of the vehicle 1 can be analyzed. The same applies to the area to the left front.

[0078] When vehicle 1 is traveling on a narrow road, the sides and front of the vehicle are near obstacles, so there is little need to resolve distant objects. However, it is better to resolve some distance to the front of the vehicle (including the right and left diagonal front). Therefore, in the example above, the vertical position of the area containing the right and left diagonal front of vehicle 1 is made higher than the vertical position of the area containing the right and left front of vehicle 1 (i.e., θ2 > θ3).

[0079] In any of the examples above, the region determination rule specifies that the right-front and left-front sides of vehicle 1 are treated as distortion mitigation targets at a higher frequency than the right front, left front, right rear, left rear, and rear front sides of vehicle 1. In scenarios where vehicle 1 is traveling on narrow roads, the probability of vehicle 1 coming into contact with objects located to its left and right front sides is high. Therefore, by treating the left-front and right-front sides of vehicle 1 as the analysis targets at a higher frequency, the processing load on control device 2 can be reduced, and appropriate analysis corresponding to the driving scenario can be performed.

[0080] In some embodiments, the control device 2 can also determine whether the shooting by the multiple fisheye cameras 44 is proceeding normally based on images acquired at the same time of movement in the same direction for the vehicle 1. For example, in a typical driving scenario, the situation to the left front of the vehicle 1 is included in both region 201L of fisheye camera 41 and region 204R of fisheye camera 44 at the same time of movement every three cycles. The control device 2 can also determine whether the shooting by fisheye camera 41 and fisheye camera 44 is proceeding normally by comparing the images of region 201L and region 204R. If the images of region 201L and region 204R are inconsistent (e.g., an object is only included in the image of one of them), the control device 2 can also determine that at least one of the shooting by fisheye camera 41 and fisheye camera 44 is not proceeding normally.

[0081] By comparing the images in region 201L and region 204R alone, it is sometimes impossible to determine which of the fisheye cameras, 41 and 44, is malfunctioning. For example, even if fisheye camera 41 malfunctions and cannot capture a specific object (e.g., a pedestrian), fisheye camera 44, which is functioning normally, can capture that object. In this case, fisheye camera 44, which can capture the object, is functioning normally. On the other hand, if there is dirt on the lens of fisheye camera 41, fisheye camera 44 will not capture the dirt. In this case, fisheye camera 44, which cannot capture the object, is functioning normally. The control device 2 can also determine which fisheye camera is malfunctioning by analyzing only the objects captured by either fisheye camera 41 or fisheye camera 44.

[0082] The control device 2 can also determine which fisheye camera's capture is not performing correctly by comparing two images in multiple directions of the vehicle. For example, as described above, the control device 2 can also determine whether at least one of the captures by fisheye camera 41 and fisheye camera 44 is not performing correctly by comparing region 201L of fisheye camera 41 with region 204R of fisheye camera 44. Furthermore, the control device 2 can determine whether at least one of the captures by fisheye camera 41 and fisheye camera 42 is performing correctly by comparing region 201R of fisheye camera 41, which respectively includes the right-facing view of vehicle 1, with region 202L of fisheye camera 42. As a result, if both the captures by fisheye camera 41 and fisheye camera 42 are normal, the control device 2 can determine that the capture by fisheye camera 44 is not performing correctly. In the above example, the left-facing and right-facing views of vehicle 1 are used, but the control device 2 can also use other tilt directions of vehicle 1.

[0083] <Summary of Implementation Methods>

[0084] <Project 1>

[0085] A control device (2) is a control device for a movable body (1) having multiple shooting devices (41-44), the multiple shooting devices including a first shooting device and a second shooting device.

[0086] The control device includes:

[0087] An image acquisition unit acquires images (300) of the external environment of the moving body from the plurality of imaging devices;

[0088] The correction unit performs distortion reduction processing on one or more regions (302) contained in the images acquired from the plurality of imaging devices to reduce image distortion; and

[0089] The recognition unit identifies the external environment of the moving object based on the image (303) after the distortion reduction processing.

[0090] When the correction unit uses a first region in the image acquired from the first imaging device that contains a state with a specific orientation of the moving body as the object of the distortion reduction processing, it also uses a second region in the image acquired from the second imaging device that contains a state with the specific orientation as the object of the distortion reduction processing.

[0091] According to this project, the external environment of moving objects can be properly identified.

[0092] <Project 2>

[0093] According to the control device described in Project 1,

[0094] The plurality of shooting devices also includes a third shooting device.

[0095] The specific direction is the first direction.

[0096] When the correction unit uses a third region in the image acquired from the first imaging device that contains a second direction different from the first direction as the object of the distortion reduction processing, it also uses a fourth region in the image acquired from the third imaging device that contains the second direction as the object of the distortion reduction processing.

[0097] According to the project, it is possible to identify the outside world from two different directions by using two shooting devices at the same time of action.

[0098] <Project 3>

[0099] According to the control device described in Project 1,

[0100] The plurality of shooting devices also includes a third shooting device and a fourth shooting device.

[0101] The specific direction is the first direction.

[0102] When the correction unit takes a third region in the image acquired from the first imaging device that contains a second direction different from the first direction as the object of the distortion reduction processing, it takes a fourth region in the image acquired from the third imaging device that contains a second direction as the object of the distortion reduction processing, and takes a fifth region in the image acquired from the second imaging device that contains a third direction different from both the first and second directions, and a sixth region in the image acquired from the fourth imaging device that contains the third direction as the object of the distortion reduction processing.

[0103] According to this project, it is possible to identify two directions separately using two shooting devices at the same moment of action.

[0104] <Project 4>

[0105] According to the control device described in Item 3, the second direction and the third direction are opposite to each other relative to the moving body.

[0106] According to the project, the shooting devices can complement each other's blind spots.

[0107] <Project 5>

[0108] According to the control device described in item 3 or 4

[0109] The first, second, third, and fourth shooting devices are divided into two groups at the first action point. The two shooting devices in one group respectively target the region containing the state of the moving body in a first direction as the object of the distortion reduction processing. The two shooting devices in the other group respectively target the region containing the state of the moving body in a fourth direction different from the first, second, and third directions as the object of the distortion reduction processing.

[0110] The first shooting device, the second shooting device, the third shooting device, and the fourth shooting device are divided into two groups at the second action time, which are different from the first action time. The two shooting devices in one group respectively take the area containing the second direction of the moving body as the object of the distortion reduction processing, and the two shooting devices in the other group respectively take the area containing the third direction of the moving body as the object of the distortion reduction processing.

[0111] According to this project, it is possible to efficiently perform analysis using four imaging devices in a specific direction.

[0112] <Project 6>

[0113] According to the control device described in any one of items 2 to 5,

[0114] The first imaging device captures images of the frontal, right-frontal, and left-frontal views of the moving object.

[0115] The second imaging device captures images of the right front, right diagonal front, and right diagonal rear of the moving object.

[0116] The third imaging device captures images of the left front, left diagonal front, and left diagonal rear of the moving object.

[0117] According to this project, it is possible to identify the external environment from all directions of a moving object.

[0118] <Project 7>

[0119] According to the control device described in any one of items 1 to 6,

[0120] Only when the moving object is in a predetermined moving scene, the correction unit will use a first region of the image acquired from the first shooting device that contains the state of the moving object in a specific direction as the object of the distortion reduction processing, and a second region of the image acquired from the second shooting device that contains the state of the moving object in the specific direction as the object of the distortion reduction processing.

[0121] According to the project, it is possible to use multiple shooting devices to identify the same direction at the same moment of action, only when necessary.

[0122] <Project 8>

[0123] According to any one of items 1 to 7, the control device further includes a determination mechanism that determines whether the shooting by the first shooting device and the shooting by the second shooting device are performed normally by comparing the image of the first region and the image of the second region.

[0124] Based on this project, the operational status of the filming device can be determined.

[0125] <Project 9>

[0126] According to the control device described in any of items 1 to 8, the specific direction is the tilting direction of the moving body.

[0127] Based on this project, it is possible to focus on analyzing the direction of tilt of the moving body.

[0128] <Project 10>

[0129] According to the control device described in any one of items 1 to 9, the plurality of shooting devices are shooting devices equipped with fisheye lenses.

[0130] According to this project, the shooting range of the shooting device can be expanded.

[0131] <Project 11>

[0132] According to the control device described in any of items 1 to 11, the moving body also has other imaging devices (40) that capture images with less distortion than those captured by the plurality of imaging devices.

[0133] The image acquisition unit acquires images of the external environment of the moving object from the other imaging devices.

[0134] The recognition unit also identifies the external environment of the moving object based on images from the other imaging devices.

[0135] According to this project, multiple types of imaging devices can be used to identify the outside world.

[0136] <Project 12>

[0137] According to the control device described in any of items 1 to 11, the moving body is a vehicle (1).

[0138] According to the project, the vehicle can properly identify its surroundings while in motion.

[0139] <Project 13>

[0140] A vehicle having the control device described in any one of items 1 to 11.

[0141] According to this project, the aforementioned effects are achieved through vehicles.

[0142] <Project 14>

[0143] A program that enables a computer to function as the various mechanisms of the control device described in any of items 1 to 11.

[0144] According to this project, the above-mentioned effects are achieved procedurally.

[0145] <Project 15>

[0146] A control method for a moving body (1) having multiple shooting devices (41-44), the multiple shooting devices including a first shooting device and a second shooting device.

[0147] The control method includes:

[0148] Image acquisition step, wherein an image of the external environment of the moving body is acquired from the plurality of imaging devices (300);

[0149] The correction step involves performing distortion reduction processing on one or more regions (302) contained in the images acquired from the plurality of imaging devices to reduce image distortion; and

[0150] In the identification step, the external environment of the moving object is identified based on the image (303) after the distortion reduction processing.

[0151] In the correction step, when a first region of the image acquired from the first imaging device that contains the condition of the moving body in a specific direction is used as the object of the distortion reduction processing, a second region of the image acquired from the second imaging device that contains the condition of the specific direction is used as the object of the distortion reduction processing.

[0152] According to this project, the external environment of moving objects can be properly identified.

[0153] This invention is not limited to the embodiments described above, and various modifications and alterations can be made within the scope of the spirit of this invention.

Claims

1. A control device for a movable body having multiple shooting devices, the multiple shooting devices including a first shooting device, a second shooting device, a third shooting device, and a fourth shooting device. Its features are, The control device includes: An image acquisition unit acquires images of the external environment of the moving object from the plurality of imaging devices; The correction unit performs distortion reduction processing on one or more regions contained in the images acquired from the plurality of imaging devices to reduce image distortion. as well as The recognition unit identifies the external environment of the moving object based on the image after distortion reduction processing. The correction unit periodically changes one or more regions that are set as the objects of the distortion reduction processing. The distortion reduction processing at the first action timing includes: a first region in the image acquired from the first imaging device that contains the first direction of the moving body; and a second region in the image acquired from the second imaging device that contains the first direction. The distortion reduction processing for the second action timing, which differs from the first action timing, targets: a third region in the image acquired from the first imaging device that includes a second direction of the moving object different from the first direction; a fourth region in the image acquired from the third imaging device that includes the second direction; a fifth region in the image acquired from the second imaging device that includes a third direction of the moving object different from both the first and second directions; and a sixth region in the image acquired from the fourth imaging device that includes the third direction. The first, second, third, and fourth shooting devices are divided into two groups at the first action point. The two shooting devices in one group respectively target the region containing the state of the moving object in the first direction as the object of the distortion reduction processing. The two shooting devices in the other group respectively target the region containing the state of the moving object in the fourth direction as the object of the distortion reduction processing. The first shooting device, the second shooting device, the third shooting device, and the fourth shooting device are divided into two groups at the second action time, which are different from the first action time. The two shooting devices in one group respectively take the area containing the second direction of the moving body as the object of the distortion reduction processing, and the two shooting devices in the other group respectively take the area containing the third direction of the moving body as the object of the distortion reduction processing.

2. The control device according to claim 1, characterized in that, The second direction and the third direction are opposite to each other relative to the moving body.

3. The control device according to claim 1, characterized in that, The first imaging device captures images of the frontal, right-frontal, and left-frontal views of the moving object. The second imaging device captures images of the right front, right diagonal front, and right diagonal rear of the moving object. The third imaging device captures images of the left front, left diagonal front, and left diagonal rear of the moving object.

4. The control device according to claim 1, characterized in that, The distortion reduction processing of the correction unit applies to both the first region and the second region only when the moving body is in a predetermined moving scenario.

5. The control device according to claim 1, characterized in that, The control device also includes a determination mechanism, which determines whether the shooting by the first shooting device and the shooting by the second shooting device are performed normally by comparing the image of the first area and the image of the second area.

6. The control device according to claim 1, characterized in that, The first direction is the tilting direction of the moving body.

7. The control device according to claim 1, characterized in that, The multiple shooting devices are shooting devices equipped with fisheye lenses.

8. The control device according to claim 1, characterized in that, The moving body also has other imaging devices that capture images with less distortion than those captured by the multiple imaging devices. The image acquisition unit acquires images of the external environment of the moving object from the other imaging devices. The recognition unit also identifies the external environment of the moving object based on images from the other imaging devices.

9. The control device according to claim 1, characterized in that, The moving body is a vehicle.

10. A vehicle comprising the control device according to any one of claims 1 to 8.

11. A storage medium storing a program for enabling a computer to function as a unit of the control device according to any one of claims 1 to 8.

12. A control method for a moving body having multiple shooting devices, wherein the multiple shooting devices include a first shooting device, a second shooting device, a third shooting device, and a fourth shooting device. It is characterized in that The control method includes: The image acquisition step involves acquiring images of the external environment of the moving object from the plurality of imaging devices. The correction step involves performing distortion reduction processing on one or more regions contained in the images acquired from the plurality of imaging devices to reduce image distortion; and In the identification step, the external environment of the moving object is identified based on the image after the distortion reduction processing. In the correction step, one or more regions that are set as the object of the distortion reduction process are periodically changed. The distortion reduction processing at the first action timing includes: a first region in the image acquired from the first imaging device that contains the first direction of the moving body; and a second region in the image acquired from the second imaging device that contains the first direction. The distortion reduction processing for the second action timing, which differs from the first action timing, targets: a third region in the image acquired from the first imaging device that includes a second direction of the moving object different from the first direction; a fourth region in the image acquired from the third imaging device that includes the second direction; a fifth region in the image acquired from the second imaging device that includes a third direction of the moving object different from both the first and second directions; and a sixth region in the image acquired from the fourth imaging device that includes the third direction. The first, second, third, and fourth shooting devices are divided into two groups at the first action point. The two shooting devices in one group respectively target the region containing the state of the moving object in the first direction as the object of the distortion reduction processing. The two shooting devices in the other group respectively target the region containing the state of the moving object in the fourth direction as the object of the distortion reduction processing. The first shooting device, the second shooting device, the third shooting device, and the fourth shooting device are divided into two groups at the second action time, which are different from the first action time. The two shooting devices in one group respectively take the area containing the second direction of the moving body as the object of the distortion reduction processing, and the two shooting devices in the other group respectively take the area containing the third direction of the moving body as the object of the distortion reduction processing.

Images