Distribution system and distribution method

Abstract

To provide a distribution system and distribution method capable of optimally editing a shot image and distributing the edited shot image.SOLUTION: A distribution system 100 includes an operation unit 30 that receives a manipulation, an acquisition unit 81 that acquires a second shot image 130 taken by a second camera 20 installed in a vehicle 3, and a first shot image 120 taken by a first camera 10 installed in the vehicle 3, an image edit unit 83 that, when the operation unit 30 receives a predetermined manipulation, performs edit processing on the second shot image 130 on the basis of the first shot image 120, and a distribution unit 85 that distributes an edited image obtained by performing edit processing on the second shot image 130.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distribution device and a distribution method. [Background technology]

[0002] BACKGROUND ART Conventionally, a driving state detection device has been proposed that accurately detects the state of a driver while a vehicle is traveling (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-152133 Summary of the Invention [Problem to be solved by the invention]

[0004] In some cases, images captured by visible light cameras are processed using masking or other processing methods, and the processed images are then distributed. However, when distributing images from inside a moving vehicle, the image may not be correctly recognized due to factors such as shaking caused by the vehicle's movement and external light, and the editing process may fail. The present invention has been made in consideration of the above background, and aims to provide a distribution device and distribution method that can distribute optimally edited images, even when distributing images taken inside a vehicle. [Means for solving the problem]

[0005] A first aspect for achieving the above object is a distribution device including an operation unit that accepts operations, an acquisition unit that acquires a first captured image taken by an infrared camera mounted on a vehicle and a second captured image taken by a visible light camera mounted on the vehicle, an image editing unit that edits the second captured image based on the first captured image when a predetermined operation is accepted by the operation unit, and a distribution unit that distributes the edited image obtained by editing the second captured image.

[0006] In the distribution device, the image editing unit may be configured to determine an editing area in the second captured image based on a result of comparison between the first captured image and the second captured image.

[0007] In the above distribution device, the image editing unit may be configured to, if the specified operation is an editing instruction for an occupant area in which an occupant of the vehicle is imaged, determine the occupant area based on a comparison result between the first captured image and the second captured image, and execute an editing process to edit the image of the determined occupant area, and if the specified operation is an editing instruction for a background area, which is an area other than the occupant area in which an occupant of the vehicle is imaged, determine the background area based on a comparison result between the first captured image and the second captured image, and execute an editing process to edit the image of the determined background area.

[0008] In the distribution device, the distribution unit may be configured to stop distribution of the second captured image or the edited image when a matching rate between the first captured image and the second captured image is equal to or less than a preset threshold value.

[0009] A second aspect for achieving the above object is to have a processor mounted on the distribution device execute an acquisition step of acquiring a first captured image taken by an infrared camera mounted on the vehicle and a second captured image taken by a visible light camera mounted on the vehicle, an editing step of editing the second captured image based on the first captured image when a predetermined operation is received by an operation unit, and a distribution step of distributing the edited image obtained by editing the second captured image. [Effects of the Invention]

[0010] According to the above-described distribution device and distribution method, even when images captured inside a vehicle are distributed, the captured images can be optimally edited and the edited captured images can be distributed. [Brief explanation of the drawings]

[0011] [Figure 1]FIG. 1 is a block diagram showing the configuration of a distribution device. [Figure 2] FIG. 2 is a diagram showing an example of the setting screen. [Figure 3] FIG. 3 is a diagram showing an example of the first captured image 0. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of a captured image. [Figure 5] FIG. 5 is a diagram showing an example of an edited image. [Figure 6] FIG. 6 is a flowchart showing the operation of the distribution device. DETAILED DESCRIPTION OF THE INVENTION

[0012] [1. Configuration of a vehicle equipped with a communication device] FIG. 1 is a block diagram showing the configuration of a distribution system 1 including a distribution device 100. The configuration of the distribution device 100 will be described with reference to FIG. The broadcasting device 100 includes a first camera 10, a second camera 20, an operation unit 30, a display unit 40, and a broadcasting processing unit 50, and is mounted on a vehicle 3. In this embodiment, the broadcasting device 100 is mounted on the vehicle 3 as a dedicated device including the first camera 10 and the second camera 20. However, a smartphone or a tablet or notebook personal computer can also be used as the broadcasting device 100. In this case, the first camera 10 and the second camera 20 are externally attached to the broadcasting device 100, and are connected to the broadcasting device 100 via a wired or wireless connection.

[0013] The first camera 10 is a driver monitor camera and is installed, for example, on an instrument panel or dashboard in front of the driver, on the top surface of a steering column, etc. The first camera 10 corresponds to an infrared camera. The first camera 10 irradiates infrared light onto a driver seated in the driver's seat, captures an image of the driver's upper body including the face, and generates a first captured image 120. The first camera 10 includes a light source that emits infrared light, a photoelectric conversion element, and a processing circuit. The photoelectric conversion element is configured with a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that is sensitive to infrared light. The processing circuit processes a signal output by the photoelectric conversion element to generate the first captured image 120. The first captured image 120 may be a still image or a moving image. In this embodiment, a case will be described in which the first captured image 120 is a moving image, and the first camera 10 captures an image at a predetermined imaging period to generate the first captured image 120 of the moving image.

[0014] The second camera 20 is a visible light camera such as a digital camera, and like the first camera 10, is installed, for example, on an instrument panel or dashboard in front of the driver, or on the top surface of a steering column. The installation positions of the first camera 10 and the second camera 20 are adjusted so that the angles of view are the same. The second camera 20 corresponds to a visible light camera. The second camera 20 includes a photoelectric conversion element such as a CCD or CMOS, and a processing circuit. The processing circuit processes a signal output by the photoelectric conversion element to generate a second captured image 130. The second captured image 130 may be a still image or a moving image. In this embodiment, a case will be described in which the second captured image 130 is a moving image, and the second camera 20 captures the image at a predetermined imaging cycle to generate the second captured image 130, which is a moving image.

[0015] The operation unit 30 includes hardware buttons and switches and receives operations from the occupant.

[0016] The display unit 40 includes a display panel such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel, and displays an image on the display panel based on display data input from a control unit 60 included in the distribution processing unit 50. The display unit 40 may also be provided with a touch sensor, so that the display unit 40 is configured as a touch panel that can accept touch operations. In this case, the touch sensor corresponds to the operation unit 30.

[0017] The distribution processing unit 50 includes a wireless communication I / F 51, an input / output I / F 53, and a control unit 60.

[0018] The wireless communication I / F 51 is a wireless communication device including an antenna, an RF (Radio Frequency) circuit, a baseband circuit, an encoder, a decoder, etc., and is connected to a communication network 5 including a mobile communication network. The wireless communication I / F 51 is connected to the server device 200 via the communication network 5. The communication network 5 includes, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, a mobile communication network such as a mobile phone network, etc.

[0019] The input / output I / F 53 is an interface through which data is input and output, and is connected to the first camera 10, the second camera 20, the operation unit 30, and the display unit 40. The input / output I / F 53 receives as input the first captured image 120 generated by the first camera 10, the second captured image 130 generated by the second camera 20, and an operation signal corresponding to an operation accepted by the operation unit 30. The input / output I / F 53 outputs the input first captured image 120, the second captured image 130, and the operation signal to the control unit 60. Display data is also input to the input / output I / F 53 from the control unit 60. The input / output I / F 53 outputs the input display data to the display unit 40.

[0020] The control unit 60 is a computer device including a storage unit 70 and a processor 80 .

[0021] The storage unit 70 includes, for example, a semiconductor memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage unit 70 stores control programs 71 such as an OS (Operating System) and application programs, and user information 73. Hereinafter, application programs will also be referred to as APP. In this embodiment, the storage unit 70 stores a distribution APP 75 that distributes the second captured image 130 as the application program. The user information 73 is information required for user authentication by the server device 200 when the second captured image 130 or the edited image 140 is uploaded to the server device 200. The user information 73 includes a user ID and a password. The storage unit 70 also temporarily stores the first captured image 120 and the second captured image 130 input from the input / output I / F 53.

[0022] The processor 80 is an arithmetic processing device that is configured with a CPU (Central Processing Unit) and an MPU (Micro Processor Unit). The processor 80 may be configured with a single processor, or may be configured with multiple processors.

[0023] The processor 80 has, as its functional configuration, an acquisition unit 81, an image editing unit 83, and a distribution unit 85. The acquisition unit 81, the image editing unit 83, and the distribution unit 85 are functions realized when the processor 80 executes the distribution APP 75.

[0024] FIG. 2 is a diagram showing an example of an APP screen 110 displayed on the display unit 40. As shown in FIG. When the occupant operates the operation unit 30 and selects the distribution APP 75, the processor 80 executes the distribution APP 75 to display the APP screen 110. When the occupant wants to mask the face or background of the second captured image 130, the occupant operates the operation unit 30 to display a setting screen 111. The setting screen 111 is an example of the APP screen 110. The setting screen 111 shown in FIG. 2 displays radio buttons 113 and 115 for selecting the area to be masked. When the occupant wants to mask the face, he / she selects the radio button 113, and when the occupant wants to mask the background, he / she selects the radio button 115.

[0025] When the acquisition unit 81 receives the settings on the setting screen 111, it acquires the first captured image 120 and the second captured image 130 from the storage unit 70. The acquisition unit 81 may also be configured to instruct the first camera 10 and the second camera 20 to capture images, and acquire the first captured image 120 captured by the first camera 10 at a predetermined imaging cycle and the second captured image 130 captured by the second camera 20 at the predetermined imaging cycle.

[0026] When the image editing unit 83 receives an editing instruction from the operation unit 30, it edits the second captured image 130 based on the first captured image 120. The image editing unit 83 determines an editing area in the second captured image 130 based on a comparison result between the first captured image 120 and the second captured image 130.

[0027] Fig. 3 shows an example of the first captured image 120, and Fig. 4 shows an example of the second captured image 130. Fig. 5 shows an edited image 140 in which a mask image 150 selected by the occupant is superimposed on an area corresponding to the face of the occupant captured in the second captured image 130. When the editing instruction is an editing operation on the person area 135 where the occupant is imaged, for example, an operation to mask the occupant's face with a mask image 150, the image editing unit 83 identifies the person area 135 where the occupant is imaged based on the comparison result between the first captured image 120 and the second captured image 130.

[0028] For example, the image editing unit 83 may identify a person region 123 in the first captured image 120 and a background region 121, which is a region other than a person, by using temperature information of the person obtained from the first captured image 120 and color information of the person obtained from the second captured image 130. Alternatively, the image editing unit 83 may create a histogram of the number of pixels versus temperature for the first captured image 120 and identify the background region 121 and the person region 123 in the first captured image 120 based on a comparison result with a threshold. The image editing unit 83 separates the first captured image 120 into the person region 123 and the background region 121, and identifies a person region 135 and the background region 131 in the second captured image 130 based on this separation result. The image editing unit 83 superimposes a mask image 150 selected by the occupant onto an area corresponding to the face of the person region 135, based on the color information of the person obtained from the second captured image 130.

[0029] Furthermore, when the editing instruction is an editing operation on the background region 131, which is a region other than the person region 135, for example, an operation to superimpose an image formed in a single color or a landscape image on the background region 131, the image editing unit 83 identifies the person region 135 in the second captured image 130 by a similar procedure, and identifies the region other than the identified person region 135 as the background region 131. The image editing unit 83 superimposes the image formed in a single color, the landscape image, or the like on the identified background region 131.

[0030] Furthermore, when the distribution unit 85 distributes the edited image 140 edited by the image editing unit 83, the image editing unit 83 may calculate the matching rate between the first captured image 120 and the second captured image 130, and if the calculated matching rate is equal to or less than a preset value, instruct the distribution unit 85 to prohibit distribution. This makes it possible to prevent an edited image 140 that has been incompletely edited from being distributed.

[0031] The distribution unit 85 acquires the user information 73 from the storage unit 70, and uploads distribution information including the acquired user information 73 and the second captured image 130 or the edited image 140 to the server device 200 via the communication network 5. The distribution unit 85 may also be configured to distribute the second captured image 130 or the edited image 140 directly to the user terminal 300 without going through the server device 200.

[0032] The server device 200 distributes content to the user terminal 300. The server device 200 receives distribution information uploaded from the distribution device 100. The server device 200 associates the second captured image 130 or the edited image 140 included in the received distribution information with the user information 73 and stores them in a storage unit of the server device 200. Then, when the server device 200 receives a content distribution request from the user terminal 300, it distributes the second captured image 130 or the edited image 140, which is the target content, to the user terminal 300.

[0033] FIG. 6 is a flowchart showing the operation of the distribution device 100. The operation of the distribution device 100 will be described with reference to the flowchart shown in FIG. The control unit 60 determines whether or not an operation to select the distribution APP 75 has been accepted by the operation unit 30 (step S1). If the operation to select the distribution APP 75 has not been accepted (step S1 / NO), the control unit 60 waits until an operation to select the distribution APP 75 is accepted.

[0034] When the control unit 60 receives an operation to select the distribution APP 75 (step S1 / YES), the control unit 60 executes the distribution APP 75 and displays the APP screen 110 on the display unit 40 (step S2).

[0035] Next, the control unit 60 determines whether or not the setting screen 111 has been selected by an operation on the APP screen 110 (step S3). If the setting screen 111 has been selected (step S3 / YES), the control unit 60 causes the display unit 40 to display the setting screen 111 (step S4) and accepts an operation to select an area to be subjected to masking. If the setting screen 111 has not been selected (step S3 / NO), the control unit 60 proceeds to the determination of step S6.

[0036] The control unit 60 determines whether or not the selection of the area to be masked is input on the setting screen 111 and the operation of the decision button 117 is accepted (step S5). If the operation of the decision button 117 is not accepted (step S5 / NO), the control unit 60 waits until the operation of the decision button 117 is accepted.

[0037] When the control unit 60 receives an operation of the decision button 117 (step S5 / YES), it determines whether or not an operation of the distribution start button displayed on the APP screen 110 has been received (step S6). When the control unit 60 has not received an operation of the distribution start button (step S6 / NO), it waits until an operation of the distribution start button is received.

[0038] When the control unit 60 receives an operation of the distribution start button (step S6 / YES), it acquires the first captured image 120 captured by the first camera 10 and the second captured image 130 captured by the second camera 20 (step S7).

[0039] Next, the control unit 60 corrects the angle of view and parallax of the first camera 10 and the second camera 20 based on the acquired first captured image 120 and second captured image 130 (step S8).

[0040] Next, the control unit 60 associates the coordinates of the first captured image 120 with the coordinates of the second captured image 130, and performs coordinate matching (step S9).

[0041] Next, the control unit 60 calculates the matching rate between the first captured image 120 and the second captured image 130 after coordinate matching, and determines whether the calculated matching rate is equal to or greater than a preset value (step S10). If the matching rate between the first captured image 120 and the second captured image 130 is lower than the set value (step S10 / NO), the control unit 60 stops the distribution (step 11) and returns to the processing of step 7. In another embodiment, the distribution may be terminated if the matching rate between the first captured image 120 and the second captured image 130 is lower than the set value.

[0042] If the matching rate between the first captured image 120 and the second captured image 130 is equal to or greater than a set value (step S10 / YES), the control unit 60 compares the first captured image 120 with the second captured image 130 to identify a person region 135 and a background region 131 in the second captured image 130 (step S12). For example, the control unit 60 uses temperature information of the person obtained from the first captured image 120 and color information of the person obtained from the second captured image 130 to identify the person region 135 in the second captured image 130 and the background region 131, which is a region other than the person.

[0043] Next, the control unit 60 determines whether the setting received on the setting screen 111 is a setting for masking an area corresponding to the occupant's face (step S13). If the setting received on the setting screen 111 is a setting for masking an area corresponding to the occupant's face (step S13 / YES), the control unit 60 superimposes a mask image 150 on the area corresponding to the face in the captured image to generate an edited image 140 (step S14). The control unit 60 uploads the generated edited image 140 to the server device 200 (step S17).

[0044] Furthermore, if the setting received on the setting screen 111 is not a setting to mask the area corresponding to the occupant's face (step S13 / NO), the control unit 60 determines whether or not the setting is a setting to mask the background area 131 (step S15 / YES). If the setting is a setting to mask the background area 131 (step S15 / YES), the control unit 60 superimposes a mask image 150 on the area corresponding to the background area 131 of the second captured image 130 to generate an edited image 140 (step S16). The control unit 60 uploads the generated edited image 140 to the server device 200 (step S17).

[0045] Furthermore, if the setting screen 111 does not accept a setting to perform mask processing on the background region 131 (step S15 / NO), the control unit 60 uploads the second captured image 130 as is to the server device 200 (step S18).

[0046] Next, the control unit 60 determines whether the distribution end button has been pressed (step S19). If the distribution end button has not been pressed (step S19 / NO), the control unit 60 returns to the processing of step S7. If the distribution end button has been pressed (step S19 / YES), the control unit 60 ends the distribution (step S20) and ends this processing flow.

[0047] 4. Other Embodiments The above-described embodiment is a preferred embodiment of the present invention, but the present invention is not limited to the above-described embodiment and various modifications are possible within the scope of the gist of the present invention. For example, the functional units of the distribution device 100 shown in FIG. 1 represent functional configurations, and the specific implementation form is not particularly limited. In other words, it is not necessary to implement hardware corresponding to each functional unit individually, and it is of course possible to implement a configuration in which a single processor executes a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in the above embodiments may be realized by hardware, or some of the functions realized by hardware may be realized by software.

[0048] The processing units in the flowchart shown in Fig. 6 are divided according to the main processing content to make the processing of distribution device 100 easier to understand, and the present invention is not limited by the manner in which the processing units are divided or the names of the processing units shown in the flowchart in Fig. 6. Furthermore, the processing of distribution device 100 can be divided into more processing units depending on the processing content, or one processing unit can be divided so that it includes more processes. Furthermore, the processing order of the above flowchart is not limited to the example shown in the figure.

[0049] 5. Configurations supported by the above embodiments The above embodiment is a specific example of the following configuration.

[0050] (Configuration 1) a first captured image captured by an infrared camera mounted on a vehicle and a second captured image captured by a visible light camera mounted on the vehicle; an image editing unit that, upon receiving a predetermined operation from the operation unit, edits the second captured image based on the first captured image; and a distribution unit that distributes the edited image obtained by editing the second captured image.

[0051] According to the distribution device of configuration 1, an edited image is generated by editing a second captured image captured by a visible light camera based on a first captured image captured by an infrared camera. Therefore, by accurately identifying a person's area in the second captured image based on the first captured image captured by the infrared camera and performing editing processing on the identified person's area, the image can be optimally edited and the edited image can be distributed.

[0052] (Configuration 2) 2. The distribution device according to configuration 1, wherein the image editing unit determines an editing area in the second captured image based on a result of comparing the first captured image with the second captured image.

[0053] According to the distribution device of configuration 2, an editing area for editing the second captured image is determined based on a comparison result between the first captured image and the second captured image, which makes it possible to determine the editing area with high accuracy.

[0054] (Configuration 3) The image editing unit, when the specified operation is an editing instruction for an occupant area in which an occupant of the vehicle is imaged, determines the occupant area based on a comparison result between the first captured image and the second captured image, and executes an editing process to edit the image of the determined occupant area; and, when the specified operation is an editing instruction for a background area, which is an area other than the occupant area in which an occupant of the vehicle is imaged, determines the background area based on a comparison result between the first captured image and the second captured image, and executes an editing process to edit the image of the determined background area.

[0055] According to the distribution device of configuration 3, when the editing instruction is for the occupant area, the occupant area of ​​the second captured image can be accurately identified based on the first captured image captured by the infrared camera, and editing of the occupant area can be performed. Also, when the editing instruction is for the background area, the background area of ​​the second captured image can be accurately identified based on the first captured image captured by the infrared camera, and editing of the background area can be performed.

[0056] (Configuration 4) The distribution device according to any one of configurations 1 to 3, wherein the distribution unit stops distribution of the second captured image or the edited image when a matching rate between the first captured image and the second captured image is equal to or less than a preset threshold value.

[0057] According to the distribution device of configuration 4, when the matching rate between the first captured image and the second captured image is equal to or less than a preset threshold, distribution of the second captured image or the edited image is stopped, thereby preventing distribution of an incompletely edited image.

[0058] (Configuration 5) The distribution method includes causing a processor mounted on a distribution device to execute an acquisition step of acquiring a first captured image taken by an infrared camera mounted on the vehicle and a second captured image taken by a visible light camera mounted on the vehicle, an editing step of editing the second captured image based on the first captured image when a predetermined operation is received by an operation unit, and a distribution step of distributing the edited image obtained by editing the second captured image.

[0059] According to the distribution method of configuration 5, it is possible to distribute the second captured image or an edited image obtained by editing the second captured image based on the first captured image, thereby improving convenience for the user. [Explanation of symbols]

[0060] 1...Distribution system, 5...Communication network, 10...First camera, 20...Second camera, 30...Operation unit, 50...Distribution processing unit, 51...Wireless communication I / F, 53...Input / output I / F, 60...Control unit, 70...Memory unit, 71...Control program, 73...User information, 75...Distribution APP, 80...Processor, 81...Acquisition unit, 83...Image editing unit, 85...Distribution unit, 100...Distribution device, 110...APP screen, 111...Setting screen, 113, 115...Radio buttons, 117...Decision button, 120...First captured image, 121...Background area, 123...Person area, 130...Second captured image, 131...Background area, 135...Person area, 140...Edited image, 150...Mask image, 200...Server device, 300...User terminal.

Claims

1. an operation unit that accepts operations; an acquisition unit that acquires a first captured image captured by an infrared camera mounted on a vehicle and a second captured image captured by a visible light camera mounted on the vehicle; an image editing unit that edits the second captured image based on the first captured image when a predetermined operation is received by the operation unit; a distribution unit that distributes an edited image obtained by editing the second captured image; A distribution device comprising:

2. The image editing unit determining an editing area in the second captured image based on a comparison result between the first captured image and the second captured image; The distribution device according to claim 1 .

3. The image editing unit When the predetermined operation is an instruction to edit an occupant area in which an occupant of the vehicle is imaged, the occupant area is determined based on a comparison result between the first captured image and the second captured image, and an editing process is executed to edit the image of the determined occupant area.

3. The distribution device according to claim 2, wherein, when the predetermined operation is an instruction to edit a background area that is an area other than an occupant area in which an occupant of the vehicle is imaged, the distribution device determines the background area based on a comparison result between the first captured image and the second captured image, and executes an editing process to edit the image of the determined background area.

4. The distribution unit When the matching rate between the first captured image and the second captured image is equal to or less than a preset threshold value, distribution of the second captured image or the edited image is stopped. The distribution device according to claim 1 .

5. A processor installed in the distribution device an acquiring step of acquiring a first captured image captured by an infrared camera mounted on the vehicle and a second captured image captured by a visible light camera mounted on the vehicle; an editing step of editing the second captured image based on the first captured image when a predetermined operation is received by the operation unit; a distribution step of distributing an edited image obtained by editing the second captured image; A delivery method that executes the above.

Images