Object detection rate calculation device, object detection rate calculation method, and program

Abstract

The purpose of the present disclosure is to determine an object detection rate by an operator of a monitoring center that monitors automatic driving.　Therefore, the present disclosure is an object detection rate calculation device that calculates an object detection rate that indicates a rate at which a person can detect a predetermined object shown in a video on the basis of video quality parameters obtained from an imaging unit installed on a mobile object that can move automatically, and distance information indicating a distance from the moving object or the imaging unit to a predetermined object.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for calculating the object detection rate by an operator at a monitoring center that monitors autonomous driving. [Background technology]

[0002] The market for autonomous driving is expanding due to advances in communication technology, improvements in the accuracy of sensors installed in vehicles, and improvements in autonomous driving control technology. Autonomous driving technology is classified into levels from Level 1 to Level 5 according to the level of autonomous driving. At Levels 1 and 2, the driver is responsible for driving, and although the autonomous driving system performs lane-keeping assist and braking, it is treated as an auxiliary function and the driver is still responsible for driving.

[0003] On the other hand, at levels 3 to 5 of autonomous driving, the car's autonomous driving system basically performs the driving, and the system is responsible for driving. At level 4, autonomous driving eliminates the need for a driver under certain conditions, making it possible to reduce accidents caused by human driving errors, but it is also possible that accidents due to system errors may occur.

[0004] Therefore, as a means of dealing with system errors, remote monitoring of footage from surveillance cameras of autonomous driving vehicles and remote operation of the vehicle by an operator at a monitoring center depending on the situation are being considered (Non-Patent Document 1). Furthermore, the Ministry of Land, Infrastructure, Transport and Tourism's guidelines (Non-Patent Document 2) state that when autonomous driving is performed, it is necessary for the vehicle to ensure safe operation by accurately identifying various hazards that may arise during operation and selecting appropriate routes and areas. Therefore, when monitoring surveillance footage transmitted from surveillance cameras of autonomous driving vehicles, operators at monitoring centers must respond appropriately when a dangerous driving situation occurs. For example, in cases where a person, animal, or object suddenly appears on the road, or an object is placed on the road that could cause a collision, the operator must appropriately identify the object that may interfere with driving and remotely control the vehicle to stop.

[0005] On the other hand, when a vehicle is driving autonomously and transmitting surveillance video to a monitoring center in real time, there is a problem that the surveillance video may be affected by fluctuations in wireless communication bandwidth, resulting in degradation of video quality. If the video quality of the surveillance video deteriorates, the operator at the monitoring center may be unable to detect objects that may interfere with driving, potentially leading to an accident. Therefore, if the video quality does not meet a certain standard during autonomous driving, it is considered necessary to take measures such as stopping the vehicle in advance. To determine the video quality standard for object detection, it is necessary to confirm the object detection rate when parameters that affect quality, such as the video bit rate, are input. ITU-T has established Recommendation P.1204 (Non-Patent Document 3) as a method for estimating the quality perceived by users from the bit rate of distributed video. However, for object detection in autonomous driving, the video quality perceived by users in general distributed video is insufficient; video quality sufficient for operators to detect objects in surveillance video is required. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Supplementary materials for the evaluation of the R&D and demonstration project for the social implementation of advanced autonomous driving and MaaS, 2021, ITS and Autonomous Driving Promotion Office, Automobile Division, Manufacturing Industries Bureau, Ministry of Industry and Trade [Non-patent document 2] Guidelines for passenger vehicle transport operators to ensure safety and convenience in unmanned autonomous driving transportation services in limited areas, Ministry of Land, Infrastructure, Transport and Tourism, Road Transport Bureau [Non-patent document 3] P.1204: Video quality assessment of streaming services over reliable transport for resolutions up to 4K , ITU-T, 2020 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in conventional surveillance video transmission in autonomous driving, the quality of the surveillance video transmitted by the vehicle may deteriorate due to changes in wireless bandwidth, etc. In such cases, the operator may continue autonomous driving despite a low object detection rate. For example, the screen of the monitoring center display is split and surveillance video from multiple vehicles is displayed simultaneously in small images, so the operator at the monitoring center may not notice the deterioration in video quality.

[0008] Furthermore, when detecting an object using surveillance footage, it is necessary to take into account the distance between the object and the vehicle. For example, the closer the object is to the vehicle, the larger it will appear in the footage. Therefore, even if the video quality is poor, the operator's detection rate may not be low.

[0009] The present invention has been made in view of the above circumstances, and has as its object to determine the detection rate of an object by an operator at a monitoring center. [Means for solving the problem]

[0010] In order to achieve the above object, the invention according to claim 1 is: using a model of object detection rate, which indicates the rate at which a person can detect a predetermined object shown in an image, estimated using the value of an image quality parameter acquired from a camera unit provided in a moving body capable of moving automatically and distance information indicating the distance from the moving body or the camera unit to the predetermined object; Based on the image quality parameters and distance information indicating a distance from the moving body or the imaging unit to a predetermined object, The aforementioned An object detection rate calculation device that calculates an object detection rate. [Effects of the Invention]

[0011] As described above, the present invention has the effect of making it possible to determine the rate at which an object is detected by an operator at a monitoring center. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is an overall configuration diagram of a communication system according to an embodiment. [Figure 2] FIG. 10 is a diagram showing an example of a model of an object detection rate estimated using a video bit rate and a distance to an object. [Figure 3] FIG. 2 is a diagram illustrating the electrical hardware configuration of each device and terminal according to the embodiment. [Figure 4] FIG. 10 is a diagram illustrating an overall configuration of a modified example of a communication system according to an embodiment. [Figure 5] 10 is a flowchart illustrating a process for estimating and presenting an object detection rate. [Figure 6] FIG. 10 is a diagram showing an example of a display screen on an operator terminal of a monitoring center. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] [System configuration of the embodiment] The configuration of a communication system according to an embodiment will be outlined with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to an embodiment. Note that the values ​​shown in Fig. 1 (1.2 Mbps, 20 m, 70%) are merely examples and are not intended to be limiting.

[0015] As shown in Figure 1, the communication system 1 is constructed by a camera 3 and various devices (an object distance setting device 2, an object detection rate calculation device 5, and a communication device 7) mounted on a moving object A such as a vehicle, and an operator terminal 8 placed on the remote monitoring center B side.

[0016] Basically, moving body A performs autonomous driving, but by having operator terminal 8 at monitoring center B display images from multiple moving bodies as shown in Fig. 6, an operator at monitoring center B can monitor the movement status of each vehicle and remotely control a specific moving body depending on the movement status. This allows the operator to remotely avoid an accident if an object or person that may interfere with driving suddenly appears while viewing the image from a specific moving body.

[0017] There may be multiple object distance setting devices 2, cameras 3, object detection rate calculation devices 5, communication devices 7, and operator terminals 8. In particular, there may be four or more cameras 3 to capture images of the front, right side, left side, and rear of the moving object A.

[0018] In this embodiment, the moving body A is assumed to be, for example, a vehicle (automobile) having a communication function and an automatic driving function. However, the moving body A is not limited to an automobile, and may be an agricultural machine such as a tractor having a communication function and an automatic driving function. Furthermore, the moving body may be a ship, an aircraft, etc., other than a vehicle.

[0019] Of these, the object distance setting device 2 is a device that sets distance information used to calculate the object detection rate by the object detection rate calculation device 5. The distance information indicates the distance between the moving object A (or camera 3) and the object, and is a predetermined fixed value such as 10 m, 20 m, or 30 m. However, the distance information may be a value specified by an operator at the monitoring center B, or may be a value set depending on the movement status of the moving object A, etc.

[0020] Camera 3 is a surveillance camera or the like mounted on moving body A, and camera 3 takes pictures of the outside of the moving body and transmits the video data obtained by taking pictures to operator terminal 8 of monitoring center B via communication device 7. Furthermore, camera 3 extracts the video bit rate from the camera's encoding information and transmits the bit rate information to object detection rate calculation device 5.

[0021] 2, the object detection rate calculation device 5 calculates the object detection rate based on the bit rate information acquired from the camera 3 and the distance information acquired from the object distance setting device 2. The object detection rate indicates the rate at which a person (a healthy person) can detect a predetermined object shown in a video. The object detection rate calculation device 5 then transmits object detection rate information indicating the calculated object detection rate to the operator terminal 8 of the monitoring center B via the communication device 7.

[0022] Figure 2 shows an example of a model for object detection rate estimated using the video bit rate and the distance to the object. The horizontal axis of the graph represents the surveillance video bit rate, and the vertical axis represents the object detection rate, with a quality estimation graph created for each distance between moving body A (or camera 3) and the object. When the distance between moving body A (or camera 3) and the object is short, the object appears larger, improving the detection rate. Furthermore, the higher the input video bit rate, the better the detection rate.

[0023] The method of calculating the object detection rate performed by the object detection rate calculation device 5 will be described in detail later.

[0024] The communication device 7 can communicate data with the operator terminal 8 via a communication network 100. The communication network 100 is constructed using a mobile network, the Internet, etc. The Internet also includes a space Internet that passes through outer space using an artificial satellite, etc.

[0025] The operator terminal 8 is a terminal such as a PC (Personal Computer) used by an operator at the monitoring center B.

[0026] With the above configuration, bit rate information transmitted from camera 3 is sent to object detection rate calculation device 5. In addition, distance information transmitted from object distance setting device 2 is also sent to object detection rate calculation device 5. Furthermore, video data transmitted from camera 3 is sent to operator terminal 8 via communication device 7. Object detection rate information indicating the object detection rate calculated by object detection rate calculation device 5 is transmitted from object detection rate calculation device 5 and sent to operator terminal 8 via communication device 7.

[0027] The object distance setting device 2 may be installed on the monitoring center B side. In this case, the distance information transmitted from the object distance setting device 2 is transmitted to the object detection rate calculation device 5 via a communication device newly installed on the monitoring center B side and then via a communication device 7 on the mobile object A side.

[0028] [Hardware configuration] Next, the electrical hardware configuration of the object distance setting device 2, the object detection rate calculation device 5, the communication device 7, and the operator terminal 8 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the electrical hardware configuration of each device and terminal according to the embodiment.

[0029] As shown in FIG. 3, the object detection rate calculation device 5 is a computer that includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, an external device connection I / F (Interface) 105, a network I / F 106, a media I / F 109, and a bus line 110.

[0030] Of these, the CPU 101 controls the overall operation of the object detection rate calculation device 5. The ROM 102 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 101. The RAM 103 is used as a work area for the CPU 101.

[0031] The SSD 104 reads or writes various data under the control of the CPU 101. Note that instead of the SSD 104, an HDD (Hard Disk Drive) may be used.

[0032] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0033] The network I / F 106 is an interface for performing data communication via a communication network N or a LAN (Local Area Network). Note that the network I / F 106 may be replaced by a wireless communication device.

[0034] The media I / F 109 controls reading and writing (storing) of data from and to a recording medium 109m such as a flash memory, etc. The recording medium 109m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0035] The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101 shown in FIG.

[0036] The object distance setting device 2, the communication device 7, and the operator terminal 8 have the same configuration as the object detection rate calculation device 5, and therefore a description thereof will be omitted.

[0037] [Modification of the system configuration of the embodiment] Here, a modified example of FIG. 1 will be described using FIG. 4. Note that the values ​​shown in FIG. 4 (1.2 Mbps, 20 m, 70%) are merely examples and are not limited to these. In the communication system 1 of FIG. 4, the object detection rate calculation device 5 on the mobile object A side is installed on the monitoring center B side, and a communication device 9 is also installed in the monitoring center B. Bit rate information transmitted from the camera 3 is sent to the object detection rate calculation device 5 via the communication device 7 and the communication device 9. In addition, distance information transmitted from the object distance setting device 2 is also sent to the object detection rate calculation device 5 via the communication device 7 and the communication device 9. Furthermore, the video data transmitted from the camera 3 is sent to the operator terminal 8 via the communication device 7. Note that the object distance setting device 2 may be provided on the monitoring center B side.

[0038] [Communication system processing or operation] Next, the processing or operation of each device in the communication system 1 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the processing for estimating and presenting the object detection rate. Fig. 6 is a diagram showing an example of a display screen on an operator terminal in the monitoring center.

[0039] S11: After the camera 3 captures the outside of the moving body A and obtains the video data, the video data is transmitted to the operator terminal.

[0040] S12: The camera 3 extracts the bit rate of the video using the encoding information of the video data, and transmits the bit rate information (for example, 1.2 Mbps) to the object detection rate calculation device 5.

[0041] S13: The object detection rate calculation device 5 uses the object detection rate model shown in FIG. 2 to calculate an object detection rate (for example, 70%) based on the bit rate information and distance information (for example, 20 m).

[0042] S14: The object detection rate calculation device 5 transmits object detection rate information indicating the object detection rate calculated by the object detection rate calculation device 5 to the operator terminal 8.

[0043] As a result, as shown in Fig. 6, the operator terminal 8 displays on the display, in a split screen, the video sent from each moving body in S11 and the object detection rate information related to the video of each moving body sent from the object detection rate calculation device 5 in S14, in association with each other. Note that, in order to associate the video with the object detection rate information, identification information such as an ID may be included in each of the video data and the object detection rate information. In this case, the camera 3 adds the identification information to the video data and bit rate information that it outputs. Then, the object detection rate calculation device 5 adds the identification information added to the bit rate information acquired from the camera 3 to the object detection rate information and outputs it.

[0044] S15: If the movement of the moving object A has not ended (NO), the process returns to step S11. On the other hand, if the movement of the moving object A has ended (YES), the process shown in FIG. 5 ends.

[0045] [Detailed calculation method for object detection rate] Next, a method for calculating the object detection rate performed by the object detection rate calculation device 5 through the process of S13 above will be described. Here, a method for calculating the object detection rate dr using the video bit rate br and the distance d between the moving body A (or camera 3) and the object will be described.

[0046] Video quality is correlated with the bit rate, and the higher the bit rate, the higher the video quality. On the other hand, once the bit rate reaches a certain level, the video quality reaches almost its upper limit, and the object detection rate does not change even at bit rates above that level. Therefore, the object detection rate is calculated using an object detection rate model in which the object detection rate increases as the bit rate increases, but the detection rate remains unchanged when the bit rate is above a predetermined value (first predetermined value). Furthermore, the object detection rate is calculated using an object detection rate model in which the object detection rate increases when the distance between moving body A (or camera 3) and the object is below a predetermined value (second predetermined value) (relatively close), and the detection rate remains above a certain value even when the bit rate is below a predetermined value (third predetermined value) (relatively low). On the other hand, the object detection rate decreases as the distance between moving body A (or camera 3) and the object increases, and is calculated using an object detection rate model in which the object detection rate becomes zero when the distance is above a predetermined value (fourth predetermined value) (too far). Using a formula that satisfies these conditions, the object detection rate is calculated using one of the following formulas. The detection rate is defined in the range of 0 to 1, and in each of the formulas below, if the object detection rate dr is 0 or less, the object detection rate is defined as 0, and if dr is 1 or more, the object detection rate is defined as 1. In each formula, a1-a5 are coefficients that differ for each formula, and e represents the base of the natural logarithm. Note that the formulas used to calculate the object detection rate are not limited to those shown below.

[0047]

number

[0048]

number

[0049]

number

[0050]

number

[0051]

number

[0052]

number

[0053]

number

[0054]

number

[0055] [Effects of the embodiment] As described above, according to this embodiment, it is possible to determine and present the object detection rate by the operator of the monitoring center B. As a result, the operator terminal 8 of the monitoring center B displays the monitoring video from each moving object in association with the object detection rate information for this monitoring video, as shown in Fig. 6, thereby eliminating the situation where the operator does not notice that the video quality of the monitoring video has deteriorated.

[0056] 〔supplement〕 As described above, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible, for example, as shown below.

[0057] (1) In the above embodiment, the camera 3 outputs bit rate information, and the object detection rate calculation device 5 calculates the object detection rate using the bit rate information, but this is not limited to this. For example, instead of bit rate information, the camera 3 may output resolution information indicating the resolution of the video or frame rate information indicating the frame rate of the video, and the object detection rate calculation device 5 may calculate the object detection rate using the resolution information or frame rate information. Note that the bit rate information, resolution information, and frame rate information are examples of parameters of video quality.

[0058] (2) Each device can be realized by a computer and a program, but this program can also be recorded on a (non-temporary) recording medium or provided via a communication network such as the Internet.

[0059] (3) The CPU 101 may be not only a single CPU, but also multiple CPUs.

[0060] [Relationship with basic application] This patent application claims priority based on International Patent Application PCT / JP2022 / 029392, filed on July 29, 2022, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0061] 1. Communication Systems A Mobile B Monitoring Center 2 Object distance setting device 3 Camera (example of a camera) 5. Object detection rate calculation device 7. Communications equipment 8 Operator Terminal 9. Communication Equipment 100 Communication Network

Claims

1. An object detection rate calculation device that uses a model of object detection rate that indicates the rate at which a person can detect a specified object shown in an image, estimated using the values ​​of image quality parameters obtained from a camera unit installed on a mobile body capable of moving automatically and distance information that indicates the distance from the mobile body or the camera unit to the specified object, and calculates the object detection rate based on the image quality parameters and distance information that indicates the distance from the mobile body or the camera unit to the specified object.

2. 2. The object detection rate calculation device according to claim 1, wherein the model of the object detection rate calculation device is a model in which the object detection rate increases as the parameter increases, but the object detection rate remains unchanged when the parameter is equal to or greater than a first predetermined value, the object detection rate increases when the distance is equal to or less than a second predetermined value, and the object detection rate remains above a certain value even when the parameter is equal to or less than a third predetermined value, and the object detection rate decreases as the distance increases, and the object detection rate becomes 0 when the distance is equal to or greater than a fourth predetermined value.

3. The object detection rate calculation device according to claim 1 , wherein the parameter is a bit rate, a resolution, or a frame rate related to the video.

4. 3. The object detection rate calculation device according to claim 1, wherein the object detection rate calculation device transmits object detection rate information indicating the calculated object detection rate to an operator terminal for remotely monitoring the movement of the moving body.

5. 3. The object detection rate calculation device according to claim 1, wherein the object detection rate calculation device is installed on the moving body or in a monitoring center where an operator terminal for remotely monitoring the movement of the moving body is installed.

6. An object detection rate calculation method that uses a model of object detection rate that indicates the rate at which a person can detect a specified object shown in an image, estimated using the values ​​of image quality parameters obtained from a camera unit provided on a mobile body capable of moving automatically and distance information indicating the distance from the mobile body or the camera unit to the specified object, and calculates the object detection rate based on the image quality parameters and distance information indicating the distance from the mobile body or the camera unit to the specified object.

7. A program causing a computer to execute the method according to claim 6.

Images