Vehicle photographing system and vehicle photographing method

Abstract

To properly move a movable camera when imaging the state during travel of a vehicle.SOLUTION: A vehicle imaging system 100 comprises: a fixed camera 12 which images a vehicle 9; a movable camera 22 which can image the state during travel of the vehicle 9 while moving together with the vehicle 9; and controllers 11, 21 which control the fixed camera 12 and the movable camera 22. The controllers 11, 21 extract a feature amount of the vehicle 9 from image data imaged by the fixed camera 12, and control at least one of a position, direction and speed of the movable camera 22 on the basis of the feature amount.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a vehicle photographing system and a vehicle photographing method. [Background technology]

[0002] A user who likes driving may have a desire to take pictures of the exterior of his / her vehicle while driving. The user can upload the images he / she has taken to, for example, a social networking service (hereinafter, referred to as "SNS") so that many people can see them. However, it is difficult for a user to take pictures of the exterior of a vehicle while driving. Therefore, a service for taking pictures of the exterior of a vehicle while driving has been proposed.

[0003] For example, Japanese Patent Application Laid-Open Publication No. 2019-121319 (Patent Document 1) discloses a vehicle photography support device that, when a license plate number of a vehicle is captured in an image captured by a photography device, determines that the image contains the vehicle, and posts the image to a social networking site. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-121319 A [Patent Document 2] JP 2019-180024 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is possible to use a movable camera to capture the state of the target vehicle while it is moving. In order to capture a video that highlights the vehicle well (so-called SNS-friendly video), the issue of how to move the movable camera can be an issue.

[0006] The present disclosure has been made to solve the above-mentioned problems, and an object of the present disclosure is to appropriately move a movable camera when photographing the situation while a vehicle is traveling. [Means for solving the problem]

[0007] (1) A vehicle photography system according to an aspect of the present disclosure includes a first camera that photographs a vehicle, a movable second camera that can photograph the vehicle while it is traveling, and a controller that controls the first and second cameras. The controller extracts vehicle features from image data photographed by the first camera, and controls at least one of the position, orientation, and speed of the second camera based on the features.

[0008] According to the above configuration (1), by using the vehicle features extracted from the image data captured by the first camera, the position and speed of the second camera can be determined according to the vehicle, and the second camera can be moved appropriately.

[0009] (2) The controller extracts the vehicle's traveling position or traveling direction as a feature, and controls at least one of the position and orientation of the second camera based on the vehicle's traveling position or traveling direction.

[0010] (3) The controller extracts a body size of the vehicle as a feature, and controls at least one of a position and an orientation of the second camera based on the body size of the vehicle.

[0011] (4) The controller extracts the vehicle's traveling speed as a feature, and controls the speed of the second camera based on the vehicle's traveling speed.

[0012] (5) The controller extracts the body size, body shape, or body color of the vehicle as a feature, and controls the speed of the second camera based on the body size, body shape, or body color of the vehicle.

[0013] (6) The controller extracts a body size of the vehicle as a feature, and further controls the zoom magnification of the second camera based on the body size of the vehicle.

[0014] (7) A method for photographing a vehicle by a computer according to another aspect of the present disclosure includes first and second steps. The first step is a step of extracting a feature amount of the vehicle from image data photographed by a first camera. The second step is a step of controlling at least one of a position, an orientation, and a speed of a movable second camera based on the feature amount.

[0015] According to the method of (7) above, as in the configuration of (1) above, the position or speed of the second camera is determined according to the feature amount of the vehicle, so that the second camera can be moved appropriately.

[0016] According to the present disclosure, a movable camera can be appropriately moved when photographing the vehicle while it is traveling. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating a schematic overall configuration of a vehicle photographing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram showing a typical hardware configuration of a fixed camera system. [Diagram 3] FIG. 1 is a block diagram showing a typical hardware configuration of a movable camera system. [Figure 4] FIG. 2 is a block diagram showing a typical hardware configuration of a server. [Diagram 5] FIG. 1 is a first diagram (perspective view) showing how a vehicle is photographed by a fixed camera system and a movable camera system in the first embodiment. [Figure 6] FIG. 2 is a second diagram (top view) showing how a vehicle is photographed by the fixed camera system and the movable camera system in the first embodiment. [Figure 7] FIG. 13 is a diagram showing an example of one frame of an identification video. [Figure 8] FIG. 1 is a diagram showing an example of one frame of a viewing video. [Figure 9] 2 is a functional block diagram showing the functional configuration of the fixed camera system and the movable camera system in the first embodiment. FIG. [Figure 10] FIG. 2 is a functional block diagram showing a functional configuration of a server. [Figure 11] 5 is a flowchart showing a processing procedure of a vehicle photographing process in the first embodiment. [Figure 12] FIG. 11 is a first diagram (perspective view) showing how a vehicle is photographed by a fixed camera system and a movable camera system in the second embodiment. [Figure 13] FIG. 13 is a second diagram (top view) showing how a vehicle is photographed by the fixed camera system and the movable camera system in the second embodiment. [Figure 14] FIG. 11 is a functional block diagram showing the functional configurations of a fixed camera system and a movable camera system in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference characters, and the description thereof will not be repeated.

[0019] [Embodiment 1] <System configuration> 1 is a diagram illustrating a schematic overall configuration of a vehicle photography system according to a first embodiment of the present disclosure. The vehicle photography system 100 includes a fixed camera system 1, a movable camera system 2, and a server 3. The fixed camera system 1, the movable camera system 2, and the server 3 are connected to each other via a network NW so as to be able to communicate with each other.

[0020] The fixed camera system 1 and the movable camera system 2 are installed, for example, near a road such as a race circuit, and capture images of a vehicle 9 (see FIG. 5) traveling on the road. Although one fixed camera system 1 and one movable camera system 2 are shown in FIG. 1, the vehicle photographing system 100 may include two or more fixed camera systems 1, or two or more movable camera systems 2.

[0021] The server 3 is, for example, a server owned by a company that provides vehicle photography services. The server 3 may be a cloud server provided by a cloud server management company. The server 3 generates images for the user to view from videos taken using the fixed camera system 1 and the movable camera system 2 (hereinafter, also referred to as "viewing images"), and provides the generated viewing images to the user. The viewing images are generally still images, but may be short videos. The user is often the driver of the vehicle 9, but is not particularly limited to this.

[0022] 2 is a block diagram showing a typical hardware configuration of the fixed camera system 1. The fixed camera system 1 includes a controller 11, a fixed camera 12, and a communication interface (IF) 13. The controller 11 includes a processor 111 and a memory 112. The components of the fixed camera system 1 are connected to each other via a bus or the like.

[0023] The processor 111 controls the overall operation of the fixed camera system 1. The memory 112 includes a read only memory (ROM), a random access memory (RAM), and a flash memory, none of which are shown. The memory 112 stores programs (an operating system and application programs) executed by the processor 111, and data (maps, tables, formulas, parameters, etc.) used by the programs. The memory 112 also temporarily stores moving images captured by the fixed camera system 1.

[0024] In this embodiment, fixed camera 12 captures a video that enables recognition of the number on the license plate attached to vehicle 9 (hereinafter, also referred to as "identification video"). Fixed camera 12 is preferably a high-sensitivity camera with a polarized lens.

[0025] The communication IF 13 is an interface for communicating with the movable camera system 2 and the server 3. The communication IF 13 is, for example, a communication module that complies with 4G (Generation) or 5G.

[0026] Fig. 3 is a block diagram showing a typical hardware configuration of movable camera system 2. Movable camera system 2 differs from fixed camera system 1 (see Fig. 2) in that movable camera system 2 includes movable camera 22 instead of fixed camera 12. The other configurations are the same.

[0027] Movable camera 22 captures a moving image (hereinafter also referred to as a "viewing moving image") used to generate a viewing image. Like fixed camera 12, movable camera 22 is preferably a high-sensitivity camera with a polarized lens.

[0028] Note that the fixed camera 12 corresponds to the "first camera" according to the present disclosure. The movable camera 22 corresponds to the "second camera" according to the present disclosure. The "first camera" being fixed is merely an example, and both the "first camera" and the "second camera" may be movable.

[0029] 4 is a block diagram showing a typical hardware configuration of the server 3. The server 3 includes a processor 31, a memory 32, an input device 33, a display 34, and a communication IF 35. The memory 32 includes a ROM 321, a RAM 322, and a HDD (Hard Disk Drive) 323. The components of the server 3 are connected to each other via a bus or the like.

[0030] The processor 31 executes various types of arithmetic processing in the server 3. The memory 32 stores programs executed by the processor 31 and data used by the programs. The memory 32 also stores data used for image processing by the server 3 and stores data that has been image-processed by the server 3. The input device 33 accepts input from an administrator of the server 3. The input device 33 is typically a keyboard and a mouse. The display 34 displays various information. The communication IF 35 is an interface for communicating with the fixed camera system 1 and the movable camera system 2.

[0031] <Vehicle photography> Fig. 5 is a first diagram (perspective view) showing how a vehicle is photographed by the fixed camera system 1 and the movable camera system 2 in embodiment 1. Fig. 6 is a second diagram (top view) showing how a vehicle is photographed by the fixed camera system 1 and the movable camera system 2 in embodiment 1.

[0032] 5 and 6, in the first embodiment, the fixed camera system 1 (fixed camera 12) captures the identification video from a direction in which the license plate can be captured. In this example, the identification video is captured from substantially the front of the vehicle 9.

[0033] Movable camera system 2 (movable camera 22) follows vehicle 9 and shoots a viewing video that highlights vehicle 9 (so-called SNS-friendly video). In this example, movable camera system 2 is installed on stage 801. This allows movable camera system 2 to be rotatable (i.e., the shooting angle can be changed). In addition, movable camera system 2 is configured to be adjustable in height on stage 801 (the shooting position can be changed, as described below).

[0034] Fig. 7 is a diagram showing an example of one frame of an identification video. As shown in Fig. 6 and Fig. 7, multiple vehicles 9 (91, 92) may appear in the identification video. Hereinafter, the vehicle that is the subject of the shooting (the vehicle about to shoot the viewing image) among the multiple vehicles will be referred to as the "target vehicle" and will be distinguished from the other vehicles.

[0035] 8 is a diagram showing an example of one frame of a viewing video. In this example, vehicle 91, which is a target vehicle, turns right at an intersection, and vehicle 92, which is not a target vehicle, goes straight through the intersection. As vehicle 91 turns right, movable camera system 2 changes the shooting position (vertical height) and angle (angle at which the camera is tilted relative to the horizontal direction) to shoot viewing video of vehicle 9 turning right at the intersection.

[0036] It should be noted that the viewing video is not required to show the license plate of the target vehicle (here, vehicle 91). However, the license plate of the target vehicle may be shown in the viewing video.

[0037] Vehicles 9 (including target vehicles) are not limited to four-wheeled vehicles as shown in Figs. 5 to 8, but may be, for example, two-wheeled vehicles (motorcycles). The license plate of a motorcycle is only attached to the rear. Therefore, it is easy to encounter a situation where the license plate cannot be photographed, making it difficult to identify which vehicle is the target vehicle.

[0038] <Functional configuration> 9 is a functional block diagram showing the functional configuration of fixed camera system 1 and movable camera system 2 in embodiment 1. Fixed camera system 1 includes an imaging unit 41, a communication unit 42, and a calculation processing unit 43. The calculation processing unit 43 includes a vehicle extraction unit 431, a license plate recognition unit 432, a matching processing unit 433, a target vehicle selection unit 434, and a feature extraction unit 435.

[0039] The image capturing unit 41 captures an identification video that enables the license plate number recognition unit 432 to recognize the number on the license plate. The image capturing unit 41 outputs the identification video to the vehicle extraction unit 431. The image capturing unit 41 corresponds to the fixed camera 12 in FIG.

[0040] The communication unit 42 performs bidirectional communication with a communication unit 51 (described later) of the movable camera system 2 and a communication unit 62 (described later) of the server 3 via the network NW. The communication unit 42 receives the license plate number of the target vehicle from the server 3. The communication unit 42 also transmits feature amounts (described later) extracted from the identification video to the movable camera system 2 and the server 3. The communication unit 42 corresponds to the communication IF 13 in FIG. 2.

[0041] The vehicle extraction unit 431 extracts vehicles (not limited to the target vehicle, but vehicles in general) from the identification video. This process is also referred to as "vehicle extraction process." For the vehicle extraction process, for example, a trained model generated by machine learning technology such as deep learning can be used. The vehicle extraction unit 431 outputs videos from which vehicles have been extracted (frames including vehicles) from the identification video to the number recognition unit 432 and also to the matching processing unit 433.

[0042] The license plate number recognition unit 432 recognizes the number of the license plate from the video in which the vehicle is extracted by the vehicle extraction unit 431. A trained model generated by a machine learning technique such as deep learning can be used for this process as well. The license plate number recognition unit 432 outputs the recognized number to the matching processing unit 433. The license plate number recognition unit 432 also outputs the recognized number to the communication unit 42. As a result, the license plate number of each vehicle is transmitted to the server 3.

[0043] The matching processing unit 433 associates the vehicle extracted by the vehicle extraction unit 431 with the number recognized by the number recognition unit 432. This processing is also described as a "matching processing". Specifically, referring again to FIG. 7, a situation in which two vehicles 91, 92 are extracted and two numbers 81, 82 are recognized will be described as an example. The matching processing unit 433 calculates the distance between the number and the vehicle for each number (the distance between the coordinates of the number on the frame and the coordinates of the vehicle). Then, the matching processing unit 433 matches the number with a vehicle with a short distance between the number. In this example, the distance between the number 81 and the vehicle 91 is shorter than the distance between the number 81 and the vehicle 92, so the matching processing unit 433 associates the number 81 with the vehicle 91. Similarly, the matching processing unit 433 associates the number 82 with the vehicle 92. The matching processing unit 433 outputs the result of the matching processing (the vehicle with the associated number) to the target vehicle selection unit 434.

[0044] The target vehicle selection unit 434 selects, as a target vehicle, a vehicle whose number matches the number of the target vehicle (received from the server 3) from among the vehicles whose numbers have been associated by the matching process. The target vehicle selection unit 434 outputs the vehicle selected as the target vehicle to the feature extraction unit 435.

[0045] The feature extraction unit 435 extracts the feature of the target vehicle by analyzing a video including the target vehicle. More specifically, the feature extraction unit 435 extracts the driving state (driving position, driving speed, driving direction, etc.) of the target vehicle and the appearance (body shape, body color, body size, etc.) of the target vehicle. In addition, the feature extraction unit 435 outputs the feature of the target vehicle to the communication unit 42. As a result, the feature of the target vehicle is transmitted to the movable camera system 2 and the server 3.

[0046] The movable camera system 2 includes a communication unit 51, an imaging unit 52, and a calculation processing unit 53. The calculation processing unit 53 includes an imaging control unit 531, a video buffer 532, and a video clipping unit 533.

[0047] The communication unit 51 performs bidirectional communication with the communication unit 42 of the fixed camera system 1 and a communication unit 62 (described later) of the server 3 via the network NW. The communication unit 51 receives features (described later) extracted from the identification video from the fixed camera system 1. The communication unit 51 also transmits the viewing video shot by the movable camera system 2 to the server 3. The communication unit 51 corresponds to the communication IF 23 in FIG. 3.

[0048] The image capturing unit 52 captures viewing video used to generate a viewing image under the control of the image capturing control unit 531. The image capturing unit 52 outputs the viewing video to the video buffer 532. The image capturing unit 52 corresponds to the movable camera 22 in FIG.

[0049] The photographing control section 531 controls the movement (position and speed of the movable camera 22) of the photographing section 52. The photographing control section 531 includes a position determination section 531A, an angle determination section 531B, a rotation speed determination section 531C, and a size adjustment section 531D.

[0050] Position determination unit 531A determines the position (height) at which movable camera 22 captures the target vehicle based on the traveling position, traveling direction, and body size of the target vehicle extracted by feature amount extraction unit 435. For example, position determination unit 531A sets the position higher when the target vehicle is a large vehicle compared to when the target vehicle is a small vehicle (such as a two-wheeled vehicle). This allows the vertical position of the target vehicle in the viewed video to be set to an appropriate position.

[0051] Angle determination unit 531B determines the angle (low angle, high angle, etc.) at which the target vehicle is photographed by movable camera 22, based on the driving position, driving direction, and body size of the target vehicle extracted by feature extraction unit 435. For example, when the target vehicle turns near movable camera 22, angle determination unit 531B can select a low angle (from a low position) to shoot a viewing video with power and realism. On the other hand, when the target vehicle travels in a direction away from movable camera 22, angle determination unit 531B can select a high angle (from a high position) to shoot a viewing video showing a series of scenes of the target vehicle's driving.

[0052] Rotation speed determination unit 531C determines the rotation speed of movable camera 22 based on the traveling position and traveling speed of the target vehicle extracted by feature extraction unit 435. In other words, rotation speed determination unit 531C controls movable camera 22 to track the target vehicle. By acquiring the traveling speed of the target vehicle in advance, rotation speed determination unit 531C can track the target vehicle by rotating movable camera 22 at an appropriate speed from the initial timing when viewing video of the target vehicle becomes available to be captured (the timing when the target vehicle enters the capturing range of movable camera 22).

[0053] The rotation speed determination unit 531C may determine the rotation speed of the movable camera 22 based on the appearance (such as the body size, body shape, or body color) of the target vehicle extracted by the feature extraction unit 435. By acquiring the appearance of the target vehicle in advance, the rotation speed determination unit 531C can distinguish the target vehicle from other vehicles and track the target vehicle more accurately.

[0054] The size adjustment unit 531D adjusts the size (zoom magnification, etc.) of the target vehicle captured by the movable camera 22 based on the body size of the target vehicle extracted by the feature extraction unit 435. This also makes it possible to set the target vehicle in the viewed video to an appropriate size.

[0055] The shooting control unit 531 may include only one, two, or three of the position determination unit 531A, the angle determination unit 531B, the rotation speed determination unit 531C, and the size adjustment unit 531D. In other words, the shooting control unit 531 may determine only one or two of the position, direction (angle), and speed (rotation speed) of the movable camera 22 based on the feature extracted by the feature extraction unit 435.

[0056] The video buffer 532 temporarily stores the viewing video. The video buffer 532 is typically a ring buffer (circular buffer) and has a circular storage area in which the beginning and end of a one-dimensional array are logically linked. Newly captured viewing videos are stored in the video buffer 532 for a predetermined time that can be stored in the storage area. Old viewing videos that exceed the predetermined time are automatically deleted from the video buffer 532.

[0057] Video cutout unit 533 cuts out a video of a predetermined time width (for example, several seconds to several tens of seconds) including the timing when the target vehicle is photographed from the viewing video stored in video buffer 532. Video cutout unit 533 outputs the cut-out viewing video to communication unit 51. As a result, the viewing video including the target vehicle is transmitted to server 3. Note that video cutout unit 533 may identify a target video in the viewing video using the feature extracted by feature extraction unit 435, and then cut out the viewing video including the target vehicle.

[0058] 10 is a functional block diagram showing the functional configuration of the server 3. The server 3 includes a storage unit 61, a communication unit 62, and a calculation processing unit 63. The storage unit 61 includes an image storage unit 611 and a registration information storage unit 612. The calculation processing unit 63 includes a vehicle extraction unit 631, a target vehicle identification unit 632, an image processing unit 633, an album creation unit 634, a web service management unit 635, and a camera system management unit 636.

[0059] The image storage unit 611 stores the viewing image obtained as a result of the arithmetic processing by the server 3. More specifically, the image storage unit 611 stores images before and after processing by the image processing unit 633, and also stores albums created by the album creation unit 634.

[0060] The registration information storage unit 612 stores registration information related to the vehicle photography service. The registration information includes personal information of a user who has applied for the vehicle photography service and vehicle information of the user. The personal information of the user includes information related to the user's identification number (ID), name, date of birth, address, telephone number, email address, etc. The vehicle information of the user includes information related to the license plate number of the vehicle. The vehicle information may further include information related to the vehicle model, year, body shape (sedan, wagon, minivan, etc.), body color, body size, etc.

[0061] The communication unit 62 performs two-way communication with the communication unit 42 of the fixed camera system 1 and the communication unit 51 (see FIG. 9) of the movable camera system 2 via the network NW. The communication unit 62 transmits the license plate number of the target vehicle to the fixed camera system 1. The communication unit 62 also receives feature amounts of the target vehicle from the fixed camera system 1 and receives viewing video including the target vehicle from the movable camera system 2. The communication unit 42 corresponds to the communication IF 35 in FIG. 4.

[0062] The vehicle extraction unit 631 extracts vehicles (not limited to the target vehicle, but vehicles in general) from the viewed video. For this process, a trained model can be used, similar to the vehicle extraction process by the vehicle extraction unit 431 of the fixed camera system 1. The vehicle extraction unit 631 outputs a video from which a vehicle has been extracted (a frame including a vehicle) to the target vehicle identification unit 632.

[0063] The target vehicle identification unit 632 identifies a target vehicle from among the vehicles extracted by the vehicle extraction unit 631 based on the feature quantities of the target vehicle (i.e., driving conditions such as driving position, driving speed, and driving direction, and external appearances such as body shape, body color, and body size). A trained model generated by machine learning techniques such as deep learning can also be used for this process. An image to be viewed is generated by identifying the target vehicle by the target vehicle identification unit 632. The image to be viewed usually includes multiple images (multiple frames that are consecutive in time). The target vehicle identification unit 632 outputs the image to be viewed to the image processing unit 633.

[0064] The image processing unit 633 processes the image to be viewed. For example, the image processing unit 633 selects the most photogenic image (so-called best shot) from among a plurality of images. Then, the image processing unit 633 performs various image corrections (trimming, color correction, distortion correction, etc.) on the selected image. The image processing unit 633 outputs the processed image to be viewed to the album creation unit 634.

[0065] The album creation unit 634 creates an album using the processed viewing images. A known image analysis technology (for example, a technology for automatically creating a photo book, slide show, etc. from images taken with a smartphone) can be used to create the album. The album creation unit 634 outputs the album to the web service management unit 635.

[0066] The web service management unit 635 provides a web service (for example, an application that can be linked to an SNS) using the album created by the album creation unit 634. This allows the user to view the created album and post desired images in the album to the SNS. Note that the web service management unit 635 may be implemented in a server separate from the server 3.

[0067] The camera system management unit 636 manages (monitors and diagnoses) the fixed camera system 1. If any abnormality (camera failure, communication failure, etc.) occurs in the fixed camera system 1 or the movable camera system 2 under its management, the camera system management unit 636 notifies the administrator of the server 3. This enables the administrator to take measures such as inspecting and repairing the camera system. Like the web service management unit 635, the camera system management unit 636 can also be implemented as a separate server.

[0068] <Processing flow> Fig. 11 is a flowchart showing the processing procedure of the vehicle photography processing in this embodiment. This flowchart is executed, for example, when a predetermined condition is satisfied or at a predetermined cycle. In the figure, the left side shows the processing by the fixed camera system 1, the center shows the processing by the movable camera system 2, and the right side shows the processing by the server 3. Each step is realized by software processing by the controller 11, the controller 21, or the processor 31, but may also be realized by hardware (electrical circuitry). Hereinafter, step is abbreviated as S.

[0069] In S11, the fixed camera system 1 extracts a vehicle by executing a vehicle extraction process on the identification video. Furthermore, the fixed camera system 1 recognizes the license plate number from the identification video from which the vehicle is extracted (S12). The fixed camera system 1 transmits the recognized license plate number to the server 3.

[0070] When the server 3 receives the number from the fixed camera system 1, it refers to the registration information to determine whether the received number is a registered number (i.e., whether the vehicle photographed by the fixed camera system 1 is the vehicle (target vehicle) of the user who applied for the vehicle photography service). If the received number is a registered number (the number of the target vehicle), the server 3 transmits the number of the target vehicle to the fixed camera system 1 and requests a viewing video (S31).

[0071] In S13, the fixed camera system 1 executes a matching process between each vehicle in the recognition video and each license plate number. Then, the fixed camera system 1 selects a vehicle having the same license plate number as the target vehicle as a corresponding vehicle from among the vehicles having license plate numbers (S14). Furthermore, the fixed camera system 1 extracts the features (driving state and appearance) of the target vehicle and transmits the extracted features to the movable camera system 2 and the server 3 (S15).

[0072] When movable camera system 2 receives the feature amount from fixed camera system 1, it determines the position, orientation (angle) and speed (rotation speed) of movable camera 22 (S21). As described in FIG. 9, movable camera system 2 can determine the position and angle at which movable camera 22 photographs the target vehicle based on the target vehicle's traveling position, traveling direction, body size, etc. Movable camera system 2 can determine the rotation speed of movable camera 22 based on the target vehicle's traveling position, traveling speed and / or appearance. Movable camera system 2 can adjust the size (zoom magnification, etc.) at which the target vehicle is photographed by movable camera 22 based on the target vehicle's body size.

[0073] In S22, the movable camera system 2 captures a viewing image while controlling the movable camera 22 at the position, orientation, and speed determined in S21. Then, the movable camera system 2 cuts out a portion including the target vehicle from the captured viewing video and transmits it to the server 3.

[0074] In S32, the server 3 generates a viewing image by executing a predetermined process on the viewing video received from the movable camera system 2. The process for generating the viewing image by the server 3 has been described in detail with reference to FIG. 10, and therefore will not be described repeatedly.

[0075] As described above, in the first embodiment, when two camera systems (fixed camera system 1 and movable camera system 2) are provided, the position, orientation (angle) and speed (rotation speed) of one of the movable camera systems 2 are determined based on features extracted from image data (identification video) captured by the other fixed camera system 1. By using the features extracted from the identification video captured by the fixed camera system 1, the position, orientation and speed of the movable camera system 2 can be determined in accordance with the target vehicle, and the movable camera system 2 can be moved appropriately.

[0076] In this embodiment, an example has been described in which the fixed camera system 1 and the movable camera system 2 share the responsibility for executing arithmetic processing such as image processing. In this example, both the controller 11 of the fixed camera system 1 and the controller 21 of the movable camera system 2 correspond to the "controller" according to the present disclosure. However, either one of the controllers 11 and 21 may be in charge of the arithmetic processing. In addition, an external server 3 may be in charge of part or all of the arithmetic processing. In other words, there is no particular limitation on whether the entity of the "controller" according to the present disclosure is in the fixed camera system 1, the movable camera system 2, or externally (the server 3).

[0077] [Embodiment 2] In the first embodiment, an example has been described in which the movable camera system 2 is configured to be rotatable. In the second embodiment, an example will be described in which the movable camera system 2 is configured to be movable (translatable).

[0078] <Vehicle photography> Fig. 12 is a first diagram (perspective view) showing how a vehicle is photographed by the fixed camera system 1 and the movable camera system 2 in embodiment 2. Fig. 13 is a second diagram (top view) showing how a vehicle is photographed by the fixed camera system 1 and the movable camera system 2 in embodiment 2.

[0079] 12 and 13, in the second embodiment, as in the first embodiment, the fixed camera system 1 captures the identification video from a direction in which the license plate can be captured (for example, the front direction). Meanwhile, the movable camera system 2A is installed on a rail 802. This allows the movable camera system 2A to move in a translating manner, that is, to capture the viewing video from just beside the vehicle 9 while moving on the rail 802. In addition, the movable camera system 2A is configured to be able to adjust its height (position) on the rail 802.

[0080] 12 and 13 show one straight rail 802 extending in a direction parallel to the road. However, the shape and number of rails 802 are appropriately set according to the video to be shot. Rail 802 may extend in a direction intersecting with the road. Two or more rails 802 may be installed. Also, the installation of rail 802 itself is not essential. Movable camera system 2A may be installed on a moving body such as a radio-controlled car or a drone.

[0081] <Functional configuration> 14 is a functional block diagram showing the functional configuration of fixed camera system 1 and movable camera system 2A in embodiment 1. Fixed camera system 1 is equivalent to fixed camera system 1 in embodiment 1 (see FIG. 9). Movable camera system 2A differs from movable camera system 2 in embodiment 1 in that it includes a shooting control unit 534 instead of shooting control unit 531. Shooting control unit 534 includes a position determination unit 534A, an angle determination unit 534B, a distance determination unit 534C, and a movement speed determination unit 534D.

[0082] Similar to position determination unit 531A, position determination unit 534A determines the position (height) at which the movable camera 22 photographs the target vehicle based on the features of the target vehicle (e.g., driving position, driving direction, and body size) extracted by feature extraction unit 435.

[0083] Similar to angle determination unit 531B, angle determination unit 534B determines the angle (low angle, high angle, etc.) at which the target vehicle is photographed by movable camera 22 based on the features of the target vehicle (e.g., driving position, driving direction, and body size) extracted by feature extraction unit 435.

[0084] The distance determination unit 534C determines the distance (distance from the movable camera 22 to the target vehicle) at which the target vehicle is photographed by the movable camera 22 based on the traveling position and traveling direction of the target vehicle extracted by the feature extraction unit 435. For example, when the traveling route of the target vehicle is divided into two or more (such as when the target vehicle enters an intersection), and a movable camera 22 is installed on each of the multiple traveling routes, it is required to determine which movable camera 22 is used and from what position the target vehicle is photographed. The distance determination unit 534C selects the movable camera 22 to be used for photographing based on the traveling position and traveling direction of the target vehicle, and determines to which position the movable camera 22 should be moved. For example, the distance determination unit 534C selects a movable camera 22 that is close to the traveling position of the target vehicle and can move in the traveling direction of the target vehicle, and moves the movable camera 22 to the vicinity of the traveling position of the target vehicle (which may be the estimated traveling position). This allows the target vehicle to be photographed using a movable camera 22 that is located at an appropriate distance from the target vehicle.

[0085] The movement speed determination unit 534D determines the movement speed (translational speed) of the movable camera 22 based on the travel speed of the target vehicle extracted by the feature extraction unit 435. In other words, the movement speed determination unit 534D controls the movable camera 22 to track (or lead) the target vehicle. By acquiring the travel speed of the target vehicle in advance, the movement speed determination unit 534D can move the movable camera 22 at an appropriate movement speed to track the target vehicle from the initial timing when the viewing video of the target vehicle can be shot (the timing when the target vehicle enters the shooting range of the movable camera 22).

[0086] The moving speed determination unit 534D may determine the moving speed of the movable camera 22 based on the appearance (such as the body size, body shape, or body color) of the target vehicle extracted by the feature extraction unit 435. By acquiring the appearance of the target vehicle in advance, the moving speed determination unit 534D can distinguish the target vehicle from other vehicles and track the target vehicle more accurately.

[0087] Although not shown, the image capture control unit 534 may include a size adjustment unit (similar to the size adjustment unit 531D), which allows the size (zoom magnification, etc.) of the target vehicle captured by the movable camera 22 to be adjusted.

[0088] The photography control unit 534 may include only one, two, or three of the position determination unit 534A, the angle determination unit 534B, the distance determination unit 534C, and the movement speed determination unit 534D. In other words, the photography control unit 534 may determine only one or two of the position (position, distance), direction (angle), and speed (movement speed) of the movable camera 22 based on the feature extracted by the feature extraction unit 435.

[0089] The flowchart showing the processing procedure for capturing an image of a vehicle in the second embodiment is similar to the flowchart showing the processing procedure for capturing an image of a vehicle in the first embodiment (see FIG. 11), and therefore detailed description thereof will not be repeated.

[0090] As described above, in the second embodiment, when two camera systems (fixed camera system 1 and movable camera system 2A) are provided, the position (position, distance), orientation (angle) and speed (moving speed) of the other movable camera system 2A are determined based on the feature amounts extracted from image data (identification video) captured by one of the fixed camera systems 1. By using the feature amounts extracted from the identification video captured by the fixed camera system 1, the position, orientation and speed of the movable camera system 2A can be determined in accordance with the target vehicle, and the movable camera system 2A can be moved appropriately.

[0091] In the first and second embodiments, the target vehicle is selected by number recognition of the identification video, but the method of selecting the target vehicle is not limited to this. The target vehicle may be selected based on the communication information of the target vehicle (identification information of a communication device mounted on the target vehicle) or based on the position information of the target vehicle (latitude and longitude of the target vehicle identified by a GPS (Global Positioning System)).

[0092] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is defined by the claims, not by the description of the embodiments described above, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0093] 100 vehicle photographing system, 1 fixed camera system, 11 controller, 111 processor, 112 memory, 12 fixed camera, 13 communication IF, 2 movable camera system, 21 controller, 211 processor, 212 memory, 22 movable camera, 23 communication IF, 3 server, 31 processor, 32 memory, 321 ROM, 322 RAM, 323 HDD, 33 input device, 34 display, 35 communication IF, 41 photographing unit, 42 communication unit, 43 calculation processing unit, 431 vehicle extraction unit, 432 number recognition unit, 433 matching processing unit, 434 target vehicle selection unit, 435 feature extraction unit, 51 communication unit, 52 photographing unit, 53 calculation processing unit, 531, 534 photographing control unit, 531A, 534A position determination unit, 531B, 534B Position determination unit, 531C rotation speed determination unit, 531D size adjustment unit, 534C distance determination unit, 534D movement speed determination unit, 532 video buffer, 533 video extraction unit, 61 memory unit, 611 image memory unit, 612 registration information memory unit, 62 communication unit, 63 calculation processing unit, 631 vehicle extraction unit, 632 target vehicle identification unit, 633 image processing unit, 634 album creation unit, 635 web service management unit, 636 camera system management unit, 801 stage, 802 rails, 81,82 numbers, 9,91,92 vehicles, NW network.

Claims

1. a first camera for photographing a vehicle; a second movable camera capable of photographing the state of the vehicle while it is traveling; a controller that controls the first and second cameras; The controller: extracting a feature amount of the vehicle including a traveling position and a traveling direction of the vehicle from image data captured by the first camera; controlling a position and an orientation of the second camera based on a traveling position and a traveling direction of the vehicle; the positions of the second camera include a high position and a low position; the orientation of the second camera includes a high angle and a low angle; The controller: selecting the low position and the low angle when the vehicle turns near the second camera, while A vehicle photographing system that selects the high position and the high angle when the vehicle travels in a direction away from the second camera.

2. The controller: A body size of the vehicle is further extracted as the feature amount; The vehicle photographing system according to claim 1 , wherein at least one of a position and an orientation of the second camera is controlled based on a body size of the vehicle.

3. the second camera is configured to be translatable on a rail; The controller: A traveling speed of the vehicle is further extracted as the feature amount; The vehicle photographing system according to claim 1 , further comprising: a translation speed of the second camera being controlled based on a traveling speed of the vehicle.

4. The vehicle photography system includes a plurality of the second cameras, The plurality of second cameras include A camera that is movable in a traveling direction of the vehicle and is close to a traveling position of the vehicle; a camera that is movable in a traveling direction of the vehicle and is farther from a traveling position of the vehicle than the closer camera; The vehicle photographing system according to claim 3 , wherein the controller controls the closer camera out of the closer camera and the farther camera such that the closer camera approaches a traveling position of the vehicle.

5. The controller: Further extracting a body size, a body shape, or a body color of the vehicle as the feature amount; The vehicle photographing system according to claim 1 , wherein the speed of the second camera is controlled based on a body size, a body shape, or a body color of the vehicle.

6. The controller: A body size of the vehicle is further extracted as the feature amount; The vehicle photographing system according to claim 1 , further comprising: a zoom magnification of the second camera being further controlled based on a body size of the vehicle.

7. A method for photographing a vehicle by a computer, comprising: extracting a feature amount of the vehicle including a traveling position and a traveling direction of the vehicle from image data captured by a first camera; and controlling a position and an orientation of a movable second camera based on a traveling position and a traveling direction of the vehicle; the positions of the second camera include a high position and a low position; the orientation of the second camera includes a high angle and a low angle; The controlling step includes: selecting the low position and the low angle when the vehicle turns near the second camera; and selecting the high position and the high angle when the vehicle travels in a direction away from the second camera.

Images