Image processing device, management device, image processing program, management program, and image processing method

Abstract

To provide an in-vehicle device, management system, and upload method that can reduce the amount of data to be uploaded. [Solution] The in-vehicle device according to the embodiment is an in-vehicle device that uploads vehicle-related information to an external device at predetermined intervals, and includes a processor that uploads trimmed information obtained by trimming a target area from image information captured by the vehicle to the external device.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a management apparatus, an image processing program, a management program, and an image processing method.

Background Art

[0002] Conventionally, there is a management system that uploads video captured by a drive recorder mounted on a vehicle to a server device. For example, in such a management system, when an impact is detected by the drive recorder, the video uploaded to the server device is transmitted to a preset destination (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, it is necessary to upload all the videos to the server device once, and there is room for improvement in reducing the amount of data to be uploaded.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an in-vehicle device, a management system, and an upload method capable of reducing the amount of data to be uploaded.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an in-vehicle device according to the present invention is an in-vehicle device that uploads information related to a vehicle to an external device at a predetermined cycle, and includes a processor that uploads trimming information obtained by trimming a target area from image information captured by the vehicle to the external device. [Effects of the Invention]

[0007] According to the present invention, the amount of data to be uploaded can be reduced. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an overview of the management system. [Figure 2] Figure 2 shows an overview of the upload method. [Figure 3] Figure 3 is a block diagram of the in-vehicle device. [Figure 4] Figure 4 is a schematic diagram of a compressed file. [Figure 5] Figure 5 shows an example of trimming information. [Figure 6] Figure 6 is a block diagram of the control system. [Figure 7] Figure 7 is a flowchart showing the processing steps performed by the in-vehicle device. [Modes for carrying out the invention]

[0009] The embodiments of the in-vehicle device, management system, and upload method disclosed herein will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the embodiments described below.

[0010] First, an overview of the in-vehicle device, management system, and upload method according to the embodiment will be described using Figures 1 and 2. Figure 1 is a diagram showing an overview of the management system. Figure 2 is a diagram showing an overview of the upload method.

[0011] As shown in Figure 1, the management system S according to this embodiment is a system that performs various analyses related to driving, such as safe driving and eco-driving by drivers. Furthermore, the management system S can be introduced, for example, to delivery companies that provide delivery vehicles such as taxis and trucks, and other businesses that own commercial vehicles.

[0012] As shown in Figure 1, the management system S according to this embodiment comprises a management device 10 and an in-vehicle device 50. The in-vehicle device 50 is, for example, a drive recorder equipped with communication functions.

[0013] The in-vehicle device 50 uploads vehicle information, including driving information related to the driving status of vehicle C and image information taken by vehicle C, to the management device 10. The image information may be, for example, information about an image taken in front of vehicle C, but it may also be information about an image taken around vehicle C or inside vehicle C (for example, the driver's seat).

[0014] Furthermore, driving information is information detected by various sensors installed on vehicle C, for example. Driving information includes, for example, the position information, vehicle speed information, acceleration information, steering information, etc. of vehicle C.

[0015] The management device 10 analyzes vehicle information acquired from each in-vehicle device 50 to evaluate the current status of each vehicle C and the driver's driving. For example, the management device 10 is configured as a cloud server that provides cloud services via a network such as the internet or a mobile phone network.

[0016] Furthermore, as shown in Figure 1, the management device 10 is connected to the driver terminal 100 and the administrator terminal 200 via a network. The driver terminal 100 is a terminal owned by the driver of each vehicle C, and in the example shown in Figure 1, it is a smartphone.

[0017] The administrator terminal 200 is a terminal owned by the administrator who manages the operating status of each vehicle C, and in the example shown in Figure 1, it is a notebook PC (Personal Computer). For example, the administrator terminal 200 notifies the administrator of the current status of each vehicle C and the evaluation results for each driver by displaying information notified from the management device 10.

[0018] Here, the outline of the information processing according to the embodiment will be described using FIG. 1. For example, the management device 10 acquires vehicle information from the in-vehicle device 50 at a predetermined cycle (step S1), and analyzes the acquired vehicle information (step S2). For example, based on each vehicle information, the management device 10 performs the current situation of each vehicle C, the driving evaluation of the driver, and the like.

[0019] Subsequently, the management device 10 notifies the administrator terminal 200 of the evaluation results regarding the current situation (for example, the current location and the driving route of the vehicle C) and the driving evaluation of each vehicle C (step S3).

[0020] For example, the management device 10 periodically performs the processes of steps S1 to S3, and notifies the driver terminal 100 of the analysis results (for example, the results of the driving evaluation) at an arbitrary timing such as at the end of business (step S4).

[0021] In such a management system S, it is not preferable from the viewpoints of data volume and communication cost to upload all the videos captured by the vehicle C. Therefore, in the upload method according to the embodiment, still images are uploaded instead of videos, and furthermore, the processing load of the in-vehicle device 50 on the still images to be uploaded is reduced.

[0022] Specifically, in the upload method according to the embodiment, the data volume of the still images to be uploaded is reduced by trimming the still images. Furthermore, in the upload method according to the embodiment, usually, by trimming the key frames of the compressed files used for video recording of the drive recorder, for example, the process of separately preparing still images for upload can be reduced.

[0023] For example, as shown in Figure 2, the upload method according to this embodiment first performs video compression on the image information input from the camera. For example, the video compression is performed using inter-frame prediction and is carried out by generating a compressed file consisting of one keyframe (I-frame) that holds all the information of the frame, and multiple P-frames that represent the difference between the keyframe or the previous frame.

[0024] Furthermore, in the upload method according to the embodiment, for example, image recognition processing is performed on the keyframe (step S12). For example, the image recognition processing here is a process for recognizing an object (for example, a vehicle in front) that appears in the keyframe.

[0025] Next, in the upload method according to the embodiment, the target area in which the object recognized by the image recognition process is shown is trimmed from the keyframes in the compressed file (step S13), and the trimmed information obtained as a result of the trimming is uploaded (step S14).

[0026] In other words, in the upload method according to this embodiment, information about the range in which the object is visible in the keyframe is uploaded, and information about the range in which the object is not visible in the keyframe is not uploaded.

[0027] Furthermore, in the upload method according to this embodiment, the keyframes used for video compression to compress and save image information are the target of the upload. Therefore, it is possible to reduce processes such as preparing image information separately for upload.

[0028] In other words, according to the upload method of the embodiment, the amount of data can be reduced by cropping the still image to be uploaded, and furthermore, the processing load related to uploading can be reduced by using keyframes for such still image.

[0029] Next, an example of the configuration of the in-vehicle device 50 will be described using Figure 3. Figure 3 is a block diagram of the in-vehicle device 50. As shown in Figure 3, the in-vehicle device 50 comprises a communication unit 51, an imaging unit 52, a storage unit 53, and a control unit (processor) 54. The in-vehicle device 50 is also connected to a sensor group 61 and a display unit 62.

[0030] The sensor group 61 includes, for example, various sensors for detecting the driving state of vehicle C. These various sensors 61 include a vehicle speed sensor, brake sensor, steering angle sensor, acceleration sensor, position sensor, obstacle detection sensor, and the like.

[0031] The display unit 62 is a touch panel display mounted on the vehicle C. For example, the display unit 62 displays video input from the in-vehicle device 50. The display unit 62 may also have a speaker and output audio input from the in-vehicle device 50.

[0032] The communication unit 51 is implemented, for example, by a NIC (Network Interface Card). The communication unit 51 is connected to a predetermined communication network in a bidirectional manner and transmits and receives information with the management device 10, etc.

[0033] The imaging unit 52 is equipped with various image sensors and images the area around the vehicle C. The in-vehicle device 50 may also be configured to include an imaging unit 52 that images the interior of the vehicle (for example, the driver).

[0034] The memory unit 53 is a memory unit composed of memory devices such as non-volatile memory, data flash, and hard disk drives, and stores various information input from the imaging unit 52 and the sensor group 61.

[0035] The control unit 54 includes an acquisition unit 55, an upload unit 56, and a trimming processing unit 57, and includes, for example, a computer and various circuits having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), a hard disk drive, input / output ports, etc.

[0036] The computer's CPU functions as the acquisition unit 55, upload unit 56, and trimming unit 57 of the control unit 54, for example, by reading and executing a program stored in ROM.

[0037] The acquisition unit 55 acquires vehicle information, including image information captured by the imaging unit 52 and various driving information input from the sensor group 61. The acquisition unit 55 also stores the acquired vehicle information in the storage unit 53.

[0038] Furthermore, the acquisition unit 55 generates a compressed file by compressing multiple image pieces of information based on the difference between before and after images, and stores it in the storage unit 53. Figure 4 is a schematic diagram of a compressed file. As shown in Figure 4, for example, the compressed file F consists of one I-frame (key frame) and multiple P-frames.

[0039] An I-frame is a frame that holds all the information of the frame, and a P-frame is a frame that holds the difference information from the previous frame. In other words, the compressed file F is a file in which information is compressed between frames. By generating the compressed file F, the acquisition unit 55 can reduce the amount of image information data stored in the storage unit 53.

[0040] Returning to the explanation of Figure 3, let's describe the upload unit 56. The upload unit 56 uploads the vehicle information acquired by the acquisition unit 55 to the management device 10 at predetermined intervals. At this time, the upload unit 56 uploads the trimmed information generated by the trimming processing unit 57, which will be described later, for the image information to be uploaded.

[0041] The vehicle information uploaded by the upload unit 56 at the time of upload includes newly obtained driving information and trimming information since the previous upload.

[0042] Furthermore, the upload unit 56 uploads vehicle information of the type and period specified by the management device 10 to the management device 10 when requested by the management device 10. In this case, for example, the upload unit 56 uploads video footage taken during the period specified by the management device 10 to the management device 10.

[0043] In other words, the upload unit 56 uploads driving information for the entire period, and for image information, it uploads still images taken at the time of upload.

[0044] The upload unit 56 then uploads video footage for a specified period to the management device 10, for example, when requested by the management device 10. In this case, the upload unit 56 may also upload the keyframes (all information related to the keyframes) before trimming to the management device 10, for example, when requested by the management device 10. By uploading the entire keyframe in this way, the management device 10 or the administrator can grasp the entire keyframe.

[0045] The trimming processing unit 57 generates trimming information by trimming the target area from the image information captured by vehicle C. For example, the trimming processing unit 57 receives keyframes from the acquisition unit 55 and determines the target area by image recognition processing on the received keyframes.

[0046] Here, we will explain a specific example of processing by the trimming processing unit 57 using Figure 5. Figure 5 is a diagram showing an example of trimming information. Here, we will explain the case where trimming information is generated from image information taken inside the vehicle, but it is also possible to generate trimming information from image information taken outside the vehicle. In addition, here we will use the driver as an example of the target object, but the target object can be changed as desired.

[0047] For example, as shown in Figure 5, the trimming processing unit 57 recognizes the object (in this case, the driver) by image recognition processing on the keyframe I. The trimming processing unit 57 sets the area in which the object is visible as the target area TR and generates trimming information T by trimming the target area TR.

[0048] In other words, trimming information T is information obtained by excluding information about areas other than the target region TR from keyframe I. Therefore, by uploading trimming information T, the amount of data to be uploaded can be reduced compared to uploading the entire keyframe I. For example, if it is determined that the target object is not visible in keyframe I, the upload process for keyframe I is skipped, and a notification is sent indicating that the target object is not visible.

[0049] Furthermore, if, for example, multiple keyframes I exist before the next upload cycle, one keyframe I may be selected based on predetermined conditions, and then trimming information T may be generated. These predetermined conditions include, for example, that the subject is captured most clearly within the keyframe I.

[0050] Next, an example of the configuration of the management device 10 will be described using Figure 6. As shown in Figure 6, the management device 10 according to this embodiment includes a communication unit 20, a storage unit 30, and a control unit 40.

[0051] The communication unit 20 is implemented, for example, by a NIC (Network Interface Card). The communication unit 20 is connected to a predetermined communication network in a bidirectional manner and transmits and receives information with the in-vehicle device 50, etc.

[0052] The storage unit 30 is a storage unit composed of storage devices such as non-volatile memory, data flash, and hard disk drives. As shown in Figure 6, the storage unit 30 is It has a vehicle information database 31 and an analysis results database 32.

[0053] The vehicle information database 31 is a database that stores vehicle information acquired from each on-board device 50. The analysis results database 32 is a database that stores the analysis results for the vehicle information acquired from each on-board device 50.

[0054] The control unit 40 comprises an acquisition unit 41, an analysis unit 42, and a request unit 43, and includes, for example, a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), a hard disk drive, input / output ports, and various circuits.

[0055] The computer's CPU functions as the acquisition unit 41, analysis unit 42, and request unit 43 of the control unit 40, for example, by reading and executing a program stored in ROM.

[0056] The acquisition unit 41 acquires vehicle information for each vehicle C from each in-vehicle device 50 at predetermined intervals, and also acquires vehicle information uploaded from the in-vehicle device 50 based on a request from the request unit 43, which will be described later. The acquisition unit 41 stores the acquired vehicle information in the vehicle information database 31.

[0057] The analysis unit 42 analyzes the vehicle information acquired by the acquisition unit 41. For example, the analysis unit 42 analyzes the current status of each vehicle C or driver. Also, as part of the analysis, the analysis unit 42 evaluates the driving of each vehicle C by its driver.

[0058] For example, the analysis results from the analysis unit 42 are stored in the analysis results database 32 and are also notified to the administrator terminal 200 (see Figure 1).

[0059] The request unit 43 requests vehicle information from the in-vehicle device 50 based on the analysis results from the analysis unit 42. The request unit 43 may request vehicle information of a specified information type and period from the in-vehicle device 50 from the administrator terminal 200 that notified the analysis results from the analysis unit 42, or it may request vehicle information autonomously. When requesting vehicle information autonomously, for example, it determines whether the vehicle information satisfies predetermined conditions and requests the vehicle information associated with those conditions from the in-vehicle device 50.

[0060] Next, the processing procedure performed by the in-vehicle device 50 according to the embodiment will be described using Figure 7. Figure 7 is a flowchart showing the processing procedure performed by the in-vehicle device 50.

[0061] As shown in Figure 7, the in-vehicle device 50 performs video compression (step S101) and then performs image recognition on the keyframe I (step S102). Subsequently, based on the results of the image recognition on the keyframe I, the in-vehicle device 50 sets the target region TR and then performs trimming (step S103).

[0062] Then, the in-vehicle device 50 uploads the trimmed information obtained by trimming the keyframe I (step S104). Then, the in-vehicle device 50 terminates the process.

[0063] As described above, the in-vehicle device 50 according to the embodiment is an in-vehicle device that uploads vehicle information relating to vehicle C to a management device 10 (an example of an external device) at predetermined intervals, and includes a control unit 54 (an example of a processor) that uploads trimming information T obtained by trimming a target area from image information captured by vehicle C to the management device 10. Therefore, according to the in-vehicle device 50 according to the embodiment, the amount of data to be uploaded can be reduced.

[0064] By the way, in the embodiment described above, the case in which the management device 10 is a server or cloud system that aggregates vehicle information from each in-vehicle device 50 has been described, but it is not limited to this. Some or all of the functions of the management device 10 may be provided in the in-vehicle device 50.

[0065] Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents. [Explanation of symbols]

[0066] 10 Management device 31 Vehicle Information Database 32 Analysis Results Database 41 Acquisition Department 42 Analysis Department 43 Request part 50 Onboard equipment 52 Imaging Department 53 Memory section 54 Control Unit 55 Acquisition Department 56 Upload Section 57 Trimming Processing Section 100 Driver terminals 200 Administrator terminals C Vehicle F Compressed files I Keyframe S Management System T Trimming Information TR Target Area

Claims

1. A compressed file is created by compressing the video captured by the camera using the difference between frames. The compressed file has keyframes that have complete images, Perform image recognition processing on the keyframe, The controller has a function that generates a cropped image of the area containing the object as cropping information for transmission. Image processing device.

2. The aforementioned trimming information is, The image is a still image, and after a selection process is performed from multiple such still images, the selected still image is transmitted as the trimming information for transmission. The image processing apparatus according to claim 1.

3. The selection process is, Choose the one that best captures the subject. The image processing apparatus according to claim 2.

4. The aforementioned controller, The trimming information to be transmitted is selected according to the information for the specified period received from the management device. The image processing apparatus according to claim 1.

5. The aforementioned controller, The trimming process is performed according to the information of the specified object received from the control device. The image processing apparatus according to claim 1.

6. The aforementioned controller, If the designated object received from the management device is not captured in the image, the management device is notified accordingly. The image processing apparatus according to claim 5.

7. The terminal device performs compression processing and receives still images trimmed from keyframes. Analyze the still image and request a compressed file based on the analysis results. Management device.

8. A compressed file is created by compressing the video captured by the camera using the difference between frames. The compressed file has keyframes that have complete images, Perform image recognition processing on the keyframe, The area containing the aforementioned object is cropped to generate an image, which is then used as cropping information for transmission. An image processing program that instructs a computer to perform image processing.

9. The terminal device performs compression processing and receives still images trimmed from keyframes. Analyze the still image and request a compressed file based on the analysis results. A management program that instructs a computer to perform a process.

10. An image processing method that generates trimming information based on a compressed file that is produced by compressing video captured by a camera using the difference between frames and has keyframes that are complete images, Image recognition processing of the object is performed on the keyframes contained in the compressed file, and an image obtained by cropping the area in the keyframes in which the object is visible is generated as the cropping information. An image processing method in which a computer performs the processing.

Images