Imaging system, management server, and image providing method

Abstract

To provide an imaging system for automatically imaging an image of a traveling vehicle driven by a user from an outside or automatically imaging an image of an inside of a vehicle.SOLUTION: A management server 1 includes: a storage part 10 that stores user information of a plurality of users including a vehicle number; an image acquisition part 12 that acquires an imaging image of a vehicle 5B from a user terminal 4A; and an identification part 13 that extracts a vehicle number from the imaging image, compares the extracted vehicle number with a vehicle number included in the user information of the plurality of users, and identifies a user corresponding to the vehicle 5B imaged by the user terminal 4A. The imaged image is stored in the storage part 10 in association with the user information of the identified user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an imaging system, a management server, and an image providing method. [Background technology]

[0002] Drive recorders that capture images of the vehicle's surroundings are becoming increasingly common. Images captured by drive recorders are often used for purposes such as replaying and verifying the cause of an accident or other trouble. In addition, recent drive recorders can automatically capture high-quality images, so the images captured by drive recorders can also be used for viewing.

[0003] While dashcams and smartphones can record the scenery from inside a moving vehicle, they cannot record the exterior of your own vehicle while it is moving. There is a demand, particularly among car enthusiasts, for a service that can automatically record the exterior of the vehicle they are driving. There has also been a demand for a service that can automatically record the interior of your own car when driving with family or friends. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-092392 [Patent Document 2] Japanese Patent Publication No. 2020-135553 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure aims to provide an imaging system that automatically captures images from the outside of a vehicle being driven by a user and automatically captures images of the interior of the vehicle, a management server, and an image provision method that provides the captured images to users. [Means for solving the problem]

[0006] A management server according to one aspect of the present disclosure includes a memory unit that stores user information of multiple users including vehicle numbers, an image acquisition unit that acquires photographed images of vehicles from an external device, and an identification unit that extracts vehicle numbers from the photographed images, compares the extracted vehicle numbers with vehicle numbers included in the user information of the multiple users, and identifies the users corresponding to the vehicles in the photographed images, and stores the photographed images in the memory unit while linking them to the user information of the identified users.

[0007] A photography system according to one aspect of the present disclosure includes a first user terminal installed in a first vehicle, which photographs a second vehicle passing by while the vehicle is traveling and generates a first photographed image; a second user terminal installed in the second vehicle, which photographs the first vehicle passing by while the vehicle is traveling and generates a second photographed image; and a management server that acquires the first photographed image from the first user terminal and acquires the second photographed image from the second user terminal, wherein the management server has a memory unit that stores user information of a plurality of users, including vehicle numbers, and an identification unit that extracts the vehicle number from the first photographed image, compares the extracted number with the vehicle number included in the user information to identify the second user corresponding to the second vehicle, extracts the vehicle number from the second photographed image, compares the extracted number with the vehicle number included in the user information to identify the first user corresponding to the first vehicle, and links the first photographed image to the user information of the second user and stores it in the memory unit, and links the second photographed image to the user information of the first user and stores it in the memory unit.

[0008] An image providing method according to one aspect of the present disclosure includes the steps of: a first user terminal installed in a first vehicle photographing a second vehicle passing by while the first vehicle is traveling, generating a first photographed image, and transmitting the first photographed image to a management server; a second user terminal installed in the second vehicle photographing the first vehicle passing by while the first vehicle is traveling, generating a second photographed image, and transmitting the second photographed image to the management server; the management server extracting a vehicle number from the first photographed image, comparing the extracted number with the vehicle number included in each of the user information of a plurality of users in a memory unit, identifying the second user corresponding to the second vehicle, and transmitting the first photographed image to the second user terminal; and the management server extracting a vehicle number from the second photographed image, comparing the extracted number with the vehicle number included in each of the user information of a plurality of users in a memory unit, identifying the first user corresponding to the first vehicle, and transmitting the second photographed image to the first user terminal. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to automatically capture images of the outside of a vehicle being driven by a user, as well as the interior of the vehicle, and to provide the captured images to the user. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of an imaging system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a diagram illustrating an example of installation of a user terminal. [Figure 3] FIG. 10 is a diagram showing an example of a captured image. [Figure 4] FIG. 10 is a diagram showing an example of a captured image. [Figure 5] FIG. 2 is a block diagram showing a schematic configuration of a management server. [Figure 6] 10 is a flowchart illustrating a method for providing a captured image. [Figure 7] FIG. 10 is a schematic configuration diagram of an imaging system according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] Fig. 1 shows a photography system according to this embodiment. In this embodiment, each user installs a user terminal 4 in his or her own vehicle 5, and when the users pass each other, they photograph each other's vehicles 5 with the user terminal 4, and provide the photographed images to the corresponding user.

[0013] The user installs a dedicated app (application program) on a user terminal 4 such as a smartphone or tablet terminal, and registers as a member of the vehicle photography service. When registering as a member, the user enters user information such as the user's name, email address, license plate number of the vehicle (user vehicle), an image of the vehicle exterior, and an image of the user's face. The entered user information is stored in the management server 1.

[0014] A user who has registered as a member installs the user terminal 4 on the upper edge of the windshield of the vehicle, as shown in FIG. 2, and drives the vehicle.

[0015] The user terminal 4 has an outer camera on the rear side and an inner camera on the liquid crystal panel side, and the outer camera captures images of the outside of the vehicle and the inner camera captures images of the interior of the vehicle (vehicle occupants).

[0016] When the user terminal 4 detects an approaching vehicle with its outer camera, it photographs the vehicle as it passes by and sends the photographed image to the management server 1. The management server 1 extracts the license plate number of the vehicle shown in the image received from the user terminal 4 and compares it with the number in the user information. If there is a match, it determines that the vehicle belongs to a member registered with the vehicle photography service and identifies the corresponding user. The management server 1 stores the photographed image, linking it to the user information of the identified user.

[0017] For example, in the example shown in Fig. 1, user A installs user terminal 4A in vehicle 5A and drives the vehicle. User B installs user terminal 4B in vehicle 5B and drives the vehicle. When vehicles 5A and 5B pass each other, user terminal 4A takes an image of vehicle 5B, and user terminal 4B takes an image of vehicle 5A.

[0018] The management server 1 receives a photographed image from the user terminal 4A, extracts the license plate number of the vehicle in the image, and if it determines that the vehicle is user B's vehicle 5B, it stores the photographed image linked to user B's user information. Similarly, the management server 1 receives a photographed image from the user terminal 4B, extracts the license plate number of the vehicle in the image, and if it determines that the vehicle is user A's vehicle 5A, it stores the photographed image linked to user A's user information. As a result, an image of the exterior of the user's vehicle while it is traveling is stored linked to the user information. For example, an image such as that shown in FIG. 3 is stored.

[0019] The timing for the in-camera of the user terminal 4 to capture the interior of the vehicle is arbitrary; for example, it may capture an image when the atmosphere inside the vehicle becomes lively. For example, the user terminal 4 may analyze the image captured by the in-camera and capture an image (release the shutter) when it detects that an occupant is smiling. The user terminal 4 may also capture audio and capture an image when the conversation becomes lively. The user terminal 4 may also capture an image when an occupant makes a predetermined gesture. The user terminal 4 transmits the image captured by the in-camera to the management server 1. The management server 1 stores the received image by linking it to user information corresponding to the user terminal 4 that sent it. For example, an image such as that shown in FIG. 4 may be stored.

[0020] The management server 1 processes the captured images. Image processing includes, for example, size correction, exposure correction, composition and angle of view correction, color correction, and gradation correction. The management server 1 may also process images captured by the rear camera to create a panning effect. For example, by applying motion blur to the background areas other than the vehicle, an image can be generated that expresses a sense of dynamism and speed, like a panning shot.

[0021] Furthermore, the management server 1 may insert a watermark into the captured image to generate an image for viewing.

[0022] The user launches the dedicated app on the user terminal 4 and checks the photographed images linked to their own user information. When the management server 1 receives a purchase application for the photographed images from the user terminal 4, it performs credit card payment or the like and transmits the photographed images without watermarks to the user terminal 4 (making them available for download).

[0023] The management server 1 may also accept purchases from the user terminal 4 for processing multiple captured images into photo books, posters, calendars, acrylic stands, key chains, badges, T-shirts, and business cards. When accepting the purchase of a photo book or the like, the management server 1 performs layout editing of the photo book or the like using multiple captured images, and transmits the data of the photo book or the like to a print system (not shown). The layout editing of the photo book or the like may be performed by the user. The photo book or the like created by the print system is delivered to the user at a later date.

[0024] In this way, according to this embodiment, it is possible to automatically photograph the outside of a vehicle being driven by a user, or to automatically photograph the interior of the vehicle, and provide the captured images to the user.

[0025] Next, the processing of the management server 1 will be described with reference to Figures 5 and 6. The management server 1 is a computer equipped with a CPU, a storage unit 10, etc., and the functions of a member registration unit 11, an image acquisition unit 12, a recognition unit 13, an image processing unit 14, an image transmission unit 15, a purchase reception unit 16, and an editing unit 17 are realized by the CPU executing a vehicle photography service provision program.

[0026] The member registration unit 11 communicates with the user terminal 4 on which the dedicated app is installed, receives user information such as the user's name, email address, vehicle license plate number, vehicle exterior image, and face image, and registers the user as a member of the vehicle photography service. The user information is stored in the storage unit 10.

[0027] The image acquisition unit 12 communicates with the user terminal 4 via a network such as the Internet, and acquires an image captured by the user terminal 4 (step S1).

[0028] The identification unit 13 uses OCR analysis to extract a vehicle number from the character information of the license plate of the vehicle in the image captured by the outer camera (step S2). The identification unit 13 compares the extracted number with the number in the user information stored in the storage unit 10, and identifies the user who owns the vehicle in the captured image (step S3). The captured image is assigned to the identified user, linked to the user information, and stored in the storage unit 10 (step S4). If the number extracted from the captured image does not match the number in the user information stored in the storage unit 10, the captured image is deleted.

[0029] Furthermore, the identification unit 13 assigns the image captured by the in-camera to the user corresponding to the user terminal 4 that sent the image, and stores the image in the storage unit 10 in association with the user information.

[0030] The method by which the identification unit 13 determines whether an image received from the user terminal 4 was taken with the outer camera or the inner camera is not limited. For example, when the user terminal 4 transmits a captured image to the management server 1, the user terminal 4 may notify the management server 1 of whether the image was taken with the outer camera or the inner camera. The identification unit 13 may also analyze the image received from the user terminal 4 and determine whether the image was taken from the outside or the inside of the vehicle. Alternatively, the identification unit 13 may analyze the Exif information of the image received from the user terminal 4 and determine whether the image was taken with the outer camera or the inner camera based on lens information or the like.

[0031] The image processing unit 14 processes the captured image by correcting the size, exposure, composition and angle of view, color, tone, etc. (step S5). The image processing unit 14 may also process the image captured by the outer camera to create a panning effect. In panning effect processing, for example, a motion blur process is performed on the background parts other than the vehicle. The direction of movement of the vehicle is detected from the orientation of the vehicle, and the blur direction is determined. The image processing unit 14 may also insert a watermark into the captured image or the processed image to generate an image for viewing. The processed image is stored in the storage unit 10, linked to user information.

[0032] The image transmission unit 15 transmits a viewing page using the photographed image or the processed image linked to the user information to the user terminal 4. The image transmission unit 15 also transmits the image purchased by the user to the user terminal 4.

[0033] The purchase acceptance unit 16 accepts purchases of photographed images and processed images from the user terminal 4 (step S6). For example, the purchase acceptance unit 16 accepts a selection of images to be purchased from the user terminal 4 and performs payment by credit card or the like. The purchase acceptance unit 16 may also accept purchases of photo books and the like.

[0034] The editing unit 17 edits a photo book or the like using multiple images linked to user information. For example, the editing unit 17 edits a photo book or the like in which images are arranged in chronological order based on the date and time the images were taken. When a photo book or the like is purchased, the editing unit 17 transmits data of the photo book or the like to the print system.

[0035] In the above embodiment, the management server 1 extracts the license plate numbers of vehicles captured in captured images and assigns them to the corresponding users. However, if the license plate numbers cannot be extracted due to vegetation in the median strip, the user's vehicle may be identified based on information other than the license plate number and the image may be assigned. For example, the user terminal 4 transmits its own location information to the management server 1 in real time. The management server 1 detects passing vehicles based on the location information of each user terminal 4 and analyzes the captured image at the time of passing. The management server 1 can identify the user's vehicle by comparing the vehicle in the captured image with the vehicle exterior image included in the user information. Alternatively, the user's vehicle may be identified by comparing the driver's facial image in the captured image with the facial image included in the user information. This method can also be applied to bicycles without license plates.

[0036] In the above embodiment, the user terminal 4 may take images with the rear camera not only of passing vehicles but also of predetermined viewpoints. For example, the user terminal 4 continuously captures (photographs) the scenery outside the vehicle with the rear camera, and if a predetermined landmark (building or landscape) is included in the captured image, the captured image including the landmark is transmitted to the management server 1. When the management server 1 receives a captured image including the predetermined landmark from the user terminal 4, the management server 1 assigns the received image to the user corresponding to the user terminal 4 that sent the image, associates it with the user information, and stores it in the storage unit 10.

[0037] The user terminal 4 may also take an image using the outer camera in response to a predetermined voice from the occupant. For example, when the user terminal 4 detects a voice from the occupant saying "Take a picture here," it takes an image using the outer camera and transmits the captured image to the management server 1. The user terminal 4 also notifies the management server 1 that the image was taken based on an instruction from the occupant. Based on the notification, the management server 1 assigns the received image to the user corresponding to the user terminal 4 that sent it, associates it with the user information, and stores it in the storage unit 10.

[0038] The user terminal 4 may also take an image using the in-camera in response to a predetermined voice from the occupant. For example, when the user terminal 4 detects a voice saying "Say cheese" from the occupant, it takes an image using the in-camera and transmits the captured image to the management server 1. The management server 1 assigns the received in-camera captured image to the user corresponding to the user terminal 4 that sent it, associates it with the user information, and stores it in the storage unit 10.

[0039] Vehicles that take passing photographs are not limited to vehicles equipped with user terminals of registered users; terminals such as smartphones may be installed in public transportation such as buses and taxis to take photographs of passing user vehicles.

[0040] As shown in Fig. 7, a photographing device 2 installed at a tourist spot or scenic spot may automatically photograph a user's vehicle 5 while it is traveling, and provide the photographed image to the user (user terminal 4). The photographing device 2 continuously transmits a signal (e.g., beacon information) related to the automatic photographing service (e.g., at time intervals of every 100 milliseconds). The radio wave range of the transmitted signal is arbitrary, for example, about 100 m.

[0041] The user terminal 4 transmits a response signal when it receives a signal from the photographing device 2. Furthermore, when it receives a signal from the photographing device 2, the user terminal 4 notifies the user by voice that the photographing device 2 is nearby and that photographing will soon be performed.

[0042] When the photographing device 2 receives the response signal from the user terminal 4, it determines that a user vehicle registered in the vehicle photographing service is approaching, and then photographs the vehicle that enters a predetermined photographing range.

[0043] The photography control device 3 is communicably connected to the photography device 2, and sets the radio wave strength of the signal emitted by the photography device 2, the photography range, the number of vehicle images to be photographed, etc. The photography control device 3 acquires photographed images from the photography device 2 and transmits the acquired photographed images to the management server 1.

[0044] The management server 1 extracts the vehicle license plate number from the captured image, searches for user information, and identifies the corresponding user. The management server 1 stores the captured image in association with the user information of the identified user. This allows the user to obtain not only exterior images of the vehicle while it is moving captured by the image capture device 2, but also exterior images of the vehicle while it is moving captured by the image capture device 2.

[0045] The camera device 2 may transmit a signal including its own identification information, and the user terminal 4 that receives this signal may notify the management server 1 of the reception of the signal and the identification information of the camera device 2 included in the signal. In this case, the management server 1 instructs the camera control device 3 to perform photography with the camera device 2 based on the identification information of the camera device 2 notified from the user terminal 4. The camera control device 3 controls the camera device 2 to perform photography processing.

[0046] When the user terminal 4 transmits its own location information to the management server 1 in real time, the management server 1 may select the image capture device 2 to which the user vehicle is approaching from the location information of the user terminal 4, and transmit the vehicle license plate number of the user vehicle to the selected image capture device 2 (the image capture control device 3 that controls the image capture device 2). The image capture device 2 may use the vehicle license plate number received from the management server 1 to detect the user vehicle that has entered the capture range and capture an image. The image capture device 2 may also measure the speed of the detected user vehicle and transmit the vehicle speed information to the management server 1 along with the captured image. The vehicle speed information can be reflected in image processing. For example, the vehicle speed information may be reflected to perform processing to create a panning effect.

[0047] The image capturing device 2 is not limited to one fixed to a pillar or a building, but may be one that moves on rails installed along the road and captures images while traveling alongside the user's vehicle.Also, the image capturing device 2 may be a drone that flies within a certain range.

[0048] In the above embodiment, an example has been described in which an image outside the vehicle is captured using a user terminal 4 such as a smartphone, but images captured by one or more cameras (on-board cameras) mounted on a vehicle for sensing the surroundings in autonomous driving or ADAS (Advanced Driver Assistance System) may also be transmitted to the management server 1. An on-board communication device of the vehicle 5 may transmit the images captured by the on-board camera (together with location information obtained by GPS) to the management server 1, or the user terminal 4 may acquire the images from the on-board camera and transmit them to the management server 1.

[0049] It should be noted that the present disclosure is not limited to the above-described embodiments as they are, and that the components can be modified and embodied in the implementation stage without departing from the spirit of the present disclosure. Furthermore, multiple components disclosed in the above-described embodiments may be appropriately combined. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0050] 1 Management Server 2. Imaging equipment 3. Shooting control device 4. User terminal 5 vehicles

Claims

1. a storage unit that stores user information of a plurality of users, including vehicle numbers; an image acquisition unit that acquires a captured image of a vehicle from an external device; an identification unit that extracts a vehicle number from the captured image, compares the extracted vehicle number with vehicle numbers included in the user information of the plurality of users, and identifies the user corresponding to the vehicle in the captured image; Equipped with The management server stores the captured image in the storage unit in association with user information of the identified user.

2. an on-board camera generates the captured image; The management server according to claim 1 , wherein the external device is an on-board communication device of a vehicle having the on-board camera, or a user terminal installed in the vehicle having the on-board camera.

3. the external device is a user terminal, the user terminal is installed in a first vehicle corresponding to a first user, and has a first camera that captures an image of a second vehicle outside the vehicle and a second camera that captures an image of an occupant inside the vehicle; the image acquisition unit acquires a first captured image by the first camera and a second captured image by the second camera from the user terminal; the identification unit extracts a vehicle number from the first captured image, compares the extracted vehicle number with vehicle numbers included in user information of the plurality of users, and identifies a second user corresponding to the second vehicle; storing the first captured image in the storage unit in association with user information of the second user; The management server according to claim 1 , wherein the second captured image is stored in the storage unit in association with user information of the first user.

4. An image processing unit that performs a motion blur process on a background portion other than the vehicle in the captured image is further provided. The management server according to claim 1 , wherein the captured image on which the motion blurring process has been performed is stored in the storage unit in association with user information of the identified user.

5. a first user terminal installed in the first vehicle, which captures an image of a second vehicle passing by while traveling, and generates a first captured image; a second user terminal that is installed in the second vehicle and captures an image of the first vehicle passing by while the second vehicle is traveling, and generates a second captured image; a management server that acquires the first captured image from the first user terminal and the second captured image from the second user terminal; Equipped with The management server a storage unit that stores user information of a plurality of users, including vehicle numbers; an identification unit that extracts a vehicle number from the first captured image, compares the extracted number with the vehicle number included in the user information to identify a second user corresponding to the second vehicle, and extracts a vehicle number from the second captured image, compares the extracted number with the vehicle number included in the user information to identify a first user corresponding to the first vehicle; and An imaging system that links the first captured image to user information of the second user and stores it in the storage unit, and links the second captured image to user information of the first user and stores it in the storage unit.

6. a step in which a first user terminal installed in a first vehicle takes an image of a second vehicle passing by while the first vehicle is traveling, generates a first captured image, and transmits the first captured image to a management server; a step in which a second user terminal installed in the second vehicle takes an image of the first vehicle passing by while the second vehicle is traveling, generates a second captured image, and transmits the second captured image to the management server; The management server extracts a vehicle number from the first captured image, compares the extracted number with the vehicle number included in each of the user information of a plurality of users stored in a storage unit, identifies a second user corresponding to the second vehicle, and transmits the first captured image to the second user terminal; The management server extracts a vehicle number from the second captured image, compares the extracted number with the vehicle number included in each of the user information of the multiple users in the storage unit, identifies the first user corresponding to the first vehicle, and transmits the second captured image to the first user terminal; An image providing method comprising:

7. a first vehicle-mounted camera mounted on the first vehicle, capturing an image of a second vehicle passing by while the first vehicle is traveling, and generating a first captured image; a second vehicle-mounted camera mounted on the second vehicle, capturing an image of the first vehicle passing by while the second vehicle is traveling, and generating a second captured image; a management server that acquires the first captured image from the first vehicle and the second captured image from the second vehicle; Equipped with The management server a storage unit that stores user information of a plurality of users, including vehicle numbers; an identification unit that extracts a vehicle number from the first captured image, compares the extracted number with the vehicle number included in the user information to identify a second user corresponding to the second vehicle, and extracts a vehicle number from the second captured image, compares the extracted number with the vehicle number included in the user information to identify a first user corresponding to the first vehicle; and An imaging system that links the first captured image to user information of the second user and stores it in the storage unit, and links the second captured image to user information of the first user and stores it in the storage unit.

8. a step of capturing an image of a second vehicle passing by a first vehicle with a first on-board camera while the first vehicle is traveling, and generating a first captured image; a step of transmitting the first captured image by the first vehicle to a management server; a step of capturing an image of the first vehicle passing by the second vehicle while the second vehicle is traveling, by a second vehicle-mounted camera, and generating a second captured image; a step of transmitting the second captured image by the second vehicle to the management server; The management server extracts a vehicle number from the first captured image, compares the extracted number with the vehicle number included in each of the user information of a plurality of users stored in a storage unit, identifies a second user corresponding to the second vehicle, and transmits the first captured image to a user terminal of the second user; The management server extracts a vehicle number from the second captured image, compares the extracted number with the vehicle number included in each of the user information of a plurality of users in the storage unit, identifies the first user corresponding to the first vehicle, and transmits the second captured image to a user terminal of the first user; An image providing method comprising:

Images