Object detection rate calculation device, object detection rate calculation method, and program

Abstract

The purpose of the present disclosure is to monitor the detection rate of an object detected by an operator of a monitoring center for automatic driving. Therefore, the present disclosure is an object detection rate calculation device that calculates an object detection rate on the basis of a parameter of image quality acquired from an imaging unit provided on a moving body capable of automatically moving and distance information indicating the distance from the moving body or the imaging unit to a predetermined object, the object detection rate indicates a ratio at which a person can detect a predetermined object displayed in the video.

Description

Technical Field

[0001] The present disclosure relates to a technology for calculating a detection rate of an object detected by an operator of a monitoring center that monitors autonomous driving. Background Art

[0002] The market size of car autonomous driving is expanding due to the development of communication technology, the improvement of the accuracy of sensors installed on cars, and the improvement of autonomous driving control technology. In autonomous driving technology, it is divided into stages from Lv.1 to Lv.5 according to the level of autonomous driving. In Lv.1 and Lv.2, the main responsible body for driving is the driver, and the autonomous driving system performs lane keeping assistance, braking, etc. to assist, and the driver is responsible for driving.

[0003] On the other hand, in Lv.3 to Lv.5 of autonomous driving, the driving is basically performed by the car's autonomous driving system, and the system is responsible for driving. In Lv.4 autonomous driving, no driver is required under certain conditions, which can reduce accidents caused by human driving errors, but it is believed that accidents caused by system errors may also occur.

[0004] Therefore, as a means of dealing with system errors, it is considered to remotely monitor images through the monitoring camera of the automatic driving, and remotely operate the car by the operator of the monitoring center according to the situation (non-patent document 1). In addition, in the guidelines of the Ministry of Land, Infrastructure, Transport and Tourism (non-patent document 2), when performing automatic driving, the car must reliably grasp the various dangers caused by driving, and ensure driving safety by setting appropriate routes and areas. Therefore, when the operator of the monitoring center monitors the images sent from the monitoring camera of the automatic driving, that is, the monitoring images, it is necessary to respond appropriately when a dangerous scene occurs during driving. For example, when a person, animal or object suddenly appears on the road, or there is an object that may be hit on the road, the operator needs to appropriately grasp these objects that will hinder driving, etc., and remotely operate the car to stop it.

[0005] On the other hand, when a car performs autonomous driving and sends monitoring images to a monitoring center in real time, there is a problem that the monitoring images may be affected by changes in the frequency band of wireless communication, causing the image quality to deteriorate. In the case where the image quality of the monitoring images deteriorates, the operator at the monitoring center cannot detect objects that will hinder driving, which may lead to accidents. Therefore, in the case where the image quality does not meet a certain standard during autonomous driving, it is considered necessary to perform processing such as stopping the car in advance. In order to determine the benchmark for the image quality used to detect objects, when parameters that affect the quality such as the bit rate of the image are input, it is necessary to confirm the object detection rate. As a method for estimating the quality perceived by users based on the bit rate of distributed images, ITU-T has formulated Advisory P.1204 (Non-Patent Document 3). On the other hand, in object detection in autonomous driving, the image quality of distributed images generally perceived by users is not enough, and it is required to achieve the image quality required for operators to detect objects from monitoring images.

[0006] Non-patent literature 1: Supplementary explanatory materials on evaluation at the end of research and development and demonstration projects for social implementation of highly automated driving, MaaS, etc. (Chinese translation: Supplementary explanatory materials on evaluation at the end of research and development and demonstration projects for social implementation of highly automated driving, MaaS, etc.), 2021, ITS and Automated Driving Promotion Office, Automotive Division, Manufacturing Industries Bureau, Ministry of Industry and Trade

[0007] Non-Patent Document 2: Guidelines for Ensuring Safety and Convenience for Passenger Car Transport Operators in Unmanned Autonomous Transportation Services in Limited Areas, Ministry of Land, Infrastructure, Transport and Tourism, Automobile Bureau

[0008] Non-patent document 3: P.1204: Video quality assessment of streaming services over reliable transport for resolutions up to 4K, ITU-T, 2020 Summary of the invention

[0009] <Problems to be Solved by the Invention>

[0010] However, in the transmission of monitoring images in the past automatic driving, the image quality of the monitoring images transmitted by the car may deteriorate due to changes in the wireless frequency band, etc. In this case, the operator may continue the automatic driving even if he detects that the detection rate of the object has decreased. For example, on the monitor of the monitoring center, the monitoring images of multiple cars are displayed simultaneously due to screen division, so the operator of the monitoring center may not notice the deterioration of the image quality.

[0011] In addition, when using surveillance images to detect objects, the distance between the object and the car needs to be considered. For example, since the closer the object is to the car, the larger it appears in the image, the higher the detection rate of the operator detecting the object may be even if the image quality deteriorates.

[0012] The present invention has been made in view of the above circumstances, and an object of the present invention is to improve the detection rate of an object detected by an operator in a monitoring center.

[0013] <Methods used to solve the problem>

[0014] In order to achieve the above-mentioned purpose, the invention according to claim 1 is an object detection rate calculation device, which calculates the object detection rate based on parameters of image quality obtained from a shooting unit set on a mobile body capable of automatic movement, and distance information indicating the distance from the mobile body or the shooting unit to a specified object, wherein the object detection rate indicates the ratio at which a person can detect a specified object displayed in an image.

[0015] <Effects of the Invention>

[0016] As described above, according to the present invention, it is possible to achieve an effect of improving the detection rate at which an operator at a monitoring center detects an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an overall configuration diagram of a communication system according to an embodiment.

[0018] Figure 2 This is a diagram showing an example of a model of an object detection rate estimated using the bit rate of a video and the distance between objects.

[0019] Figure 3 It is an electrical hardware configuration diagram of each device and terminal according to an implementation manner.

[0020] Figure 4 It is an overall configuration diagram of a modified example of the communication system according to the embodiment.

[0021] Figure 5 : is a flowchart showing a process for estimating and presenting an object detection rate.

[0022] Figure 6 This is a diagram showing an example of a display screen of an operator terminal in a monitoring center. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present invention will be described using the drawings.

[0024] [System Configuration of Embodiment]

[0025] use Figure 1 The configuration of the communication system according to the embodiment is described in outline. Figure 1 is an overall configuration diagram of a communication system according to an embodiment. Note that Figure 1 The values ​​shown (1.2 Mbps, 20 m, 70%) are just examples and are not limited to these.

[0026] like Figure 1 As shown, the communication system 1 is composed of a camera 3 installed on a mobile body A such as a vehicle, various devices (object distance setting device 2, object detection rate calculation device 5 and communication device 7), and an operator terminal 8 set on the remote monitoring center B side.

[0027] Basically, the mobile body A performs the automatic driving, but the operator terminal 8 of the monitoring center B is Figure 6 As shown, by displaying images from multiple moving objects, an operator at monitoring center B can monitor the movement status of each vehicle and remotely operate a specific moving object according to the movement status. Thus, when viewing images from a specified moving object, if an object or person that would hinder driving suddenly appears, the operator can remotely prevent an accident.

[0028] Note that there may be a plurality of object distance setting devices 2, cameras 3, object detection rate calculation devices 5, communication devices 7, and operator terminals 8. In particular, in order to respectively capture the front, right side, left side, and rear of the moving body A, four or more cameras 3 may be provided.

[0029] In this embodiment, the mobile body A is assumed to be, for example, a vehicle (car) having a communication function and an automatic driving function. However, the mobile body A is not limited to a car, and may also be an agricultural machine such as a tractor having a communication function and an automatic driving function. In addition, in addition to a vehicle, the mobile body may also be a ship, an aircraft, etc.

[0030] The object distance setting device 2 is a device for setting distance information used to calculate the object detection rate by the object detection rate calculation device 5. The distance information indicates the distance between the moving body A (or the camera 3) and the object, and is a predetermined fixed value such as 10m, 20m, 30m, etc. However, the distance information may be a value specified by an operator of the monitoring center B, or may be a value set according to the movement status of the moving body A, etc.

[0031] The camera 3 is a surveillance camera installed on the moving body A, etc. The camera 3 captures the exterior of the moving body and transmits the image data obtained by the capture to the operator terminal 8 of the monitoring center B via the communication device 7. Furthermore, the camera 3 extracts the bit rate of the image from the encoding information of the camera and transmits the bit rate information to the object detection rate calculation device 5.

[0032] The object detection rate calculation device 5 uses Figure 2 The object detection rate model shown calculates the object detection rate based on the bit rate information obtained from the camera 3 and the distance information obtained from the object distance setting device 2. The object detection rate indicates the rate at which a person (healthy person) can detect a specified object shown in an image. Then, the object detection rate calculation device 5 sends the object detection rate information indicating the calculated object detection rate to the operator terminal 8 of the monitoring center B via the communication device 7.

[0033] Figure 2 This is a diagram showing an example of a model for object detection rate estimation using the bit rate of an image and the distance between an object. The horizontal axis of the diagram represents the bit rate of the surveillance image, and the vertical axis represents the detection rate of the object. A quality estimation diagram is generated according to the distance between the moving object A (or camera 3) and the object. When the distance between the moving object A (or camera 3) and the object is short, the object is displayed larger, so the detection rate is improved. In addition, the larger the bit rate of the input image, the higher the detection rate.

[0034] Note that a detailed calculation method of the object detection ratio performed by the object detection ratio calculation device 5 will be described later.

[0035] The communication device 7 can perform data communication with the operator terminal 8 via the communication network 100. The communication network 100 is composed of a mobile network, the Internet, etc. The Internet also includes a space Internet via outer space using artificial satellites or the like.

[0036] The operator terminal 8 is a terminal such as a PC (Personal Computer) used by an operator of the monitoring center B.

[0037] According to the above configuration, the bit rate information transmitted from the camera 3 is transmitted to the object detection rate calculation device 5. In addition, the distance information transmitted from the object distance setting device 2 is also transmitted to the object detection rate calculation device 5. And, the image data transmitted from the camera 3 is transmitted to the operator terminal 8 via the communication device 7. The object detection rate information indicating the object detection rate calculated by the object detection rate calculation device 5 is transmitted from the object detection rate calculation device 5 and transmitted to the operator terminal 8 via the communication device 7.

[0038] Note that the object distance setting device 2 may be provided on the monitoring center B side. In this case, the distance information sent from the object distance setting device 2 is transmitted to the object detection rate calculation device 5 via the communication device newly provided on the monitoring center B side and then via the communication device 7 on the moving body A side.

[0039] [Hardware Configuration]

[0040] Next, use Figure 3 The electrical hardware configuration of the object distance setting device 2 , the object detection rate calculation device 5 , the communication device 7 , and the operator terminal 8 will be described. Figure 3 It is an electrical hardware configuration diagram of each device and terminal according to an implementation manner.

[0041] like Figure 3 As shown, the object detection rate calculation device 5 is a computer and includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, an external device connection I / F (Interface) 105, a network I / F 106, a medium I / F 109 and a bus 110.

[0042] The CPU 101 controls the overall operation of the object detection rate calculation device 5 . The ROM 102 stores programs such as IPL (Initial Program Loader) for driving the CPU 101 . The RAM 103 is used as a work area for the CPU 101 .

[0043] The SSD 104 reads or writes various data according to the control of the CPU 101. Note that a HDD (Hard Disk Drive) may be used instead of the SSD 104.

[0044] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0045] The network I / F 106 is an interface for performing data communication via a communication network N or a LAN (Local Area Network). Note that the network I / F 106 may be replaced by a wireless communication device.

[0046] The medium I / F 109 controls reading or writing (storing) of data on a recording medium 109 m such as a flash memory, etc. The recording medium 109 m also includes a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), and the like.

[0047] Bus 110 is used to electrically connect Figure 3 The address bus, data bus, etc. of each component such as the CPU 101 shown.

[0048] Note that the object distance setting device 2 , the communication device 7 , and the operator terminal 8 have the same configuration as the object detection rate calculation device 5 , and therefore description thereof is omitted.

[0049] [Modification of System Configuration of Embodiment]

[0050] Here, use Figure 4 right Figure 1 Note that Figure 4 The values ​​shown (1.2Mbps, 20m, 70%) are examples and are not limited to these. Figure 4 In the communication system 1, the object detection rate calculation device 5 on the mobile body A side is set on the monitoring center B side, and the communication device 9 is set on the monitoring center B side. The bit rate information sent from the camera 3 is transmitted to the object detection rate calculation device 5 via the communication device 7 and the communication device 9. In addition, the distance information sent from the object distance setting device 2 is also transmitted to the object detection rate calculation device 5 via the communication device 7 and the communication device 9. And, the image data sent from the camera 3 is transmitted to the operator terminal 8 via the communication device 7. Note that the object distance setting device 2 can be set on the monitoring center B side.

[0051] [The process or operation of a communication system]

[0052] Next, use Figure 5 and Figure 6 The processing or operation of each device of the communication system 1 will be described. Figure 5 : is a flowchart showing a process for estimating and presenting an object detection rate. Figure 6 This is a diagram showing an example of a display screen of an operator terminal in a monitoring center.

[0053] S11: After the camera 3 captures the exterior of the moving object A and obtains image data, the camera 3 transmits the image data to the operator terminal.

[0054] S12 : The camera 3 extracts the bit rate of the image using the encoding information of the image data, and sends the bit rate information (eg, 1.2 Mbps) to the object detection rate calculation device 5 .

[0055] S13: Object detection rate calculation device 5 uses Figure 2 The object detection rate model shown calculates the object detection rate (eg, 70%) based on the bit rate information and the distance information (eg, 20 m).

[0056] S14 : The object detection rate calculation device 5 transmits object detection rate information indicating the object detection rate calculated by the object detection rate calculation device 5 to the operator terminal 8 .

[0057] Therefore, if Figure 6 As shown, the operator terminal 8 associates the image sent from each moving body through S11 with the object detection rate information related to the image of each moving body sent from the object detection rate calculation device 5 through S14, and displays it on the display through screen division. Note that in order to associate the image with the object detection rate information, the image data and the object detection rate information may each include identification information such as an ID. In this case, the camera 3 adds identification information to the output image data and bit rate information. Then, the object detection rate calculation device 5 adds the identification information added to the bit rate information obtained from the camera 3 to the object detection rate information and outputs it.

[0058] S15: If the movement of the moving body A has not been completed ("No"), the process returns to S11. On the other hand, if the movement of the moving body A has been completed ("Yes"), the process ends. Figure 5 Processing shown.

[0059] [Detailed calculation method of object detection rate]

[0060] Next, the object detection rate calculation method executed by the object detection rate calculation device 5 in the process of S13 is described. Here, a method of obtaining the object detection rate dr by using the bit rate br of the video and the distance d between the moving object A (or the camera 3) and the object is described.

[0061] Image quality is related to the bit rate, and the higher the bit rate, the higher the image quality. On the other hand, when the bit rate reaches a certain value or more, the image quality almost reaches the upper limit, and even if the bit rate is above this, the object detection rate does not change. Therefore, the following object detection rate model is used for calculation: the object detection rate increases as the bit rate increases, and when the bit rate is above a specified value (first specified value), the detection rate does not change. In addition, the following object detection rate model is used for calculation: when the distance between the mobile body A (or camera 3) and the object is less than a specified value (second specified value) (when the distance is relatively close), the object detection rate increases, and even when the bit rate is less than a specified value (third specified value) (when the bit rate is relatively low), the detection rate is above a certain value. On the other hand, the following object detection rate model is used for calculation: when the object detection rate decreases as the distance between the mobile body A (or camera 3) and the object becomes longer, and when the distance is more than a specified value (fourth specified value) (when the distance is too far), the object detection rate becomes 0. Using a mathematical formula that satisfies these conditions, the object detection rate is calculated by any of the following mathematical formulas. Note that the detection rate is defined in the range from 0 to 1. In the following formulas, when the object detection rate dr is below 0, the object detection rate is defined as 0, and when dr is above 1, the object detection rate is defined as 1. In each formula, a1 to a5 are coefficients that are different for each formula, and e represents the base of the natural logarithm. Note that the formula for calculating the object detection rate is not limited to the mathematical formula shown below.

[0062] [Number 1]

[0063]

[0064] [Number 2]

[0065]

[0066] [Number 3]

[0067]

[0068] [Number 4]

[0069]

[0070] [Number 5]

[0071]

[0072] [Number 6]

[0073]

[0074] [Number 7]

[0075]

[0076] [Number 8]

[0077]

[0078] Among them, A and x0 in formula 8 can be expressed as follows.

[0079] A=-c1×log(c2+br)+c3

[0080] x0=b1×log(b 2+ br)+b3

[0081] Alternatively, A can also be expressed as follows.

[0082] A=c1×exp(-(c2×x0)-c3)+c4

[0083] Here, a1, b1-b3 and c1-c4 in (Formula 8) are all coefficients.

[0084] [Effects of Embodiment]

[0085] As described above, according to this embodiment, the detection rate of objects detected by the operator of the monitoring center B can be obtained and presented. Figure 6 As shown, the operator terminal 8 of the monitoring center B displays the monitoring images from each moving object in association with the object detection rate information of the monitoring images, thereby eliminating the situation where the operator fails to notice the deterioration of the image quality of the monitoring images.

[0086] [Replenish]

[0087] As mentioned above, the present invention is not limited to the above-mentioned embodiment, and various changes and applications are possible, for example, as shown below.

[0088] (1) In the above embodiment, the camera 3 outputs the bit rate information and the object detection rate calculation device 5 uses the bit rate information to calculate the object detection rate, but the present invention is not limited to this. For example, the camera 3 may output resolution information indicating the resolution of the image or frame rate information indicating the frame rate of the image instead of the bit rate information, and the object detection rate calculation device 5 may use the resolution information or the frame rate information to calculate the object detection rate. Note that the bit rate information, the resolution information, and the frame rate information are examples of parameters of image quality.

[0089] (2) Each device can be realized by a computer and a program, but the program may be recorded in a (non-temporary) recording medium or provided via a communication network such as the Internet.

[0090] (3) The CPU 101 may be not only a single one but also multiple ones.

[0091] [Relationship with basic application]

[0092] This patent application claims priority based on international patent application PCT / JP2022 / 029392 filed on July 29, 2022, and the entire contents of international patent application PCT / JP2022 / 029392 are incorporated into this application.

[0093] Description of Reference Numerals

[0094] 1 Communication system

[0095] A Moving body

[0096] B Monitoring Center

[0097] 2 Object distance setting device

[0098] 3 Camera (an example of a photography unit)

[0099] 5 Object detection rate calculation device

[0100] 7 communication devices

[0101] 8 Operator Terminal

[0102] 9 communication devices

[0103] 100 communication network.

Claims

1. An object detection rate calculation device, which calculates an object detection rate based on a parameter of image quality obtained from a shooting unit provided on a mobile body capable of automatic movement and distance information indicating the distance from the mobile body or the shooting unit to a specified object, wherein the object detection rate indicates the ratio at which a person can detect a specified object shown in an image.

2. The object detection rate calculation device according to claim 1, wherein: The object detection rate calculation means calculates the object detection rate based on the parameter and the distance information using a model of the object detection rate that represents the relationship between the value of the parameter and the object detection rate.

3. The object detection rate calculation device according to claim 2, wherein: The model is the following model: The object detection rate increases as the parameter increases, and when the parameter is greater than a first specified value, the object detection rate no longer changes; and When the distance is less than a second predetermined value, the object detection rate increases, and even when the parameter is less than a third predetermined value, the object detection rate reaches a certain value or more; and The object detection rate decreases as the distance increases, and when the distance is equal to or greater than a fourth predetermined value, the object detection rate becomes zero.

4. The object detection rate calculation device according to any one of claims 1 to 3, wherein: The parameter is a bit rate or a resolution or a frame rate associated with the image.

5. The object detection rate calculation device according to any one of claims 1 to 3, wherein: The object detection rate calculation device transmits object detection rate information indicating the calculated object detection rate to an operator terminal for remotely monitoring the movement of the moving body.

6. The object detection rate calculation device according to any one of claims 1 to 3, wherein: The object detection rate calculation device is installed on the moving body, or is installed in a monitoring center provided with an operator terminal for remotely monitoring the movement of the moving body.

7. A method for calculating object detection rate, wherein: An object detection rate is calculated based on parameters of image quality acquired from a shooting unit provided on a mobile body capable of moving automatically, and distance information indicating the distance from the mobile body or the shooting unit to a specified object. The object detection rate indicates the ratio at which a person can detect a specified object displayed in an image.

8. A program causing a computer to execute the method according to claim 7.

Images