Object detection rate calculation device, object detection rate calculation method, and program
The object detection rate calculation device addresses the challenge of fluctuating communication band quality in autonomous driving by calculating detection rates based on image quality and speed, ensuring operators can effectively detect objects, thereby enhancing safety in autonomous driving systems.
Patent Information
- Application Number
- PCT/JP2023/034203
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-22
- Filing Date
- 2023-09-21
- Publication Date
- 2024-11-28
AI Technical Summary
In autonomous driving systems, fluctuations in wireless communication band quality can lead to deteriorated video quality, making it difficult for monitoring center operators to detect objects in time, potentially causing accidents, especially when bit rates change dynamically and vehicle speeds vary, affecting the object detection rate.
An object detection rate calculation device that calculates the percentage of observers able to detect objects based on current and past image quality parameters, distance information, and speed, using models to account for bit rate and speed fluctuations, ensuring accurate detection even under varying conditions.
The solution enables monitoring center operators to accurately determine the object detection rate, allowing for timely intervention and enhancing safety by considering dynamic bit rates and speeds, thus reducing the risk of accidents due to poor video quality.
Smart Images

Figure JP2023034203_28112024_PF_FP_ABST
Abstract
Description
Object detection rate calculation device, object detection rate calculation method, and program
[0001] The present disclosure relates to a technology for calculating the object detection rate by an operator at a monitoring center that monitors autonomous driving.
[0002] The market for autonomous driving is expanding due to advances in communication technology, improvements in the accuracy of sensors installed in vehicles, and improvements in autonomous driving control technology. Autonomous driving technology is classified into levels from Level 1 to Level 5 according to the level of autonomous driving. At Levels 1 and 2, the driver is responsible for driving, and although the autonomous driving system performs lane-keeping assist and braking, it is treated as an auxiliary function and the driver is still responsible for driving.
[0003] On the other hand, at levels 3 to 5 of autonomous driving, the car's autonomous driving system basically performs the driving, and the system is responsible for driving. At level 4, autonomous driving eliminates the need for a driver under certain conditions, making it possible to reduce accidents caused by human driving errors, but it is also possible that accidents due to system errors may occur.
[0004] As a means of dealing with system errors, remote monitoring of footage from surveillance cameras of autonomous driving vehicles and remote operation of the vehicle by an operator at a monitoring center depending on the situation are being considered (Non-Patent Document 1). Furthermore, the Ministry of Land, Infrastructure, Transport and Tourism's guidelines (Non-Patent Document 2) state that when autonomous driving is performed, the vehicle must ensure safe operation by accurately identifying various hazards that may arise during operation and selecting appropriate routes and areas. Therefore, when monitoring surveillance footage transmitted from surveillance cameras of autonomous driving vehicles, operators at monitoring centers must respond appropriately when a dangerous driving situation occurs. For example, in cases where a person, animal, or object suddenly appears on the road, or an object is placed on the road that could cause a collision, the operator must appropriately identify the object that may interfere with driving and remotely control the vehicle to stop.
[0005] On the other hand, when a vehicle is autonomously driving and transmitting surveillance video to a monitoring center in real time, there is a problem that the surveillance video may be affected by fluctuations in wireless communication bandwidth, resulting in degradation of video quality. If the video quality of the surveillance video deteriorates, the operator at the monitoring center may be unable to detect objects that may interfere with driving, potentially leading to an accident. Therefore, if the video quality does not meet certain standards during autonomous driving, it is considered necessary to take measures such as stopping the vehicle in advance. To determine the video quality standard for object detection, it is necessary to confirm the object detection rate (the percentage of observers who actually detect an object when multiple observers watch the surveillance video) when parameters that affect quality, such as the video bit rate, are input. ITU-T Recommendation P.1204 (Non-Patent Document 3) has been established as a method for estimating the quality perceived by users from the bit rate of distributed video. However, for object detection in autonomous driving, the video quality perceived by users in general distributed video is insufficient; video quality required for operators to detect objects in surveillance video is required.
[0006] However, in conventional surveillance video transmission in autonomous driving, the quality of the surveillance video transmitted by the vehicle may deteriorate due to changes in wireless bandwidth, etc. In such cases, the operator may continue autonomous driving despite a low object detection rate. For example, the screen of the operator terminal at the monitoring center is split and surveillance video from multiple vehicles is displayed simultaneously in small images, so the operator at the monitoring center may not notice the deterioration in video quality.
[0007] Furthermore, when detecting an object using surveillance footage, it is necessary to take into account the distance between the object and the vehicle. For example, the closer the object is to the vehicle, the larger it will appear in the footage. Therefore, even if the video quality is poor, the operator's detection rate may not be low.
[0008] As described above, there is a need for a technique for estimating the detection rate of an object according to the distance to the object.
[0009] Supplementary explanatory materials for the terminal evaluation of the research and development and demonstration project for the social implementation of advanced automated driving and MaaS, 2021, Ministry of Industry, Manufacturing Industries Bureau, Automobile Division, ITS and Autonomous Driving Promotion Office, Guidelines for passenger vehicle transport operators to ensure safety and convenience in unmanned autonomous driving transportation services in limited areas, Ministry of Land, Infrastructure, Transport and Tourism, Road Transport Bureau, P.1204: Video quality assessment of streaming services over reliable transport for resolutions up to 4K, ITU-T, 2020
[0010] When transmitting video in real time, it is expected that the bit rate will change dynamically according to changes in the wireless bandwidth. In order to estimate the object detection rate in such an environment, an estimation technique that can handle bit rate fluctuations is required. For example, if the bit rate is low at a certain time T, an operator at the monitoring center may not be able to detect an object X [m] away at that moment, but if the video was played back at a higher bit rate at an earlier time, it is possible that the operator could have detected the object at that position at an instant before time T.
[0011] Furthermore, in autonomous driving, the speed of a vehicle is not always constant but fluctuates. The faster the vehicle is traveling, the faster the background and other objects in the surveillance footage move. When there is a large (fast) amount of movement in the footage, encoding the footage becomes difficult, which can result in a decrease in the quality of the footage and a decrease in the object detection rate, even when the footage is transmitted from the vehicle to the monitoring center at the same bit rate. Furthermore, when the vehicle is traveling at a high speed, the distance the vehicle travels between detecting an object and the operator's response becomes longer, making it necessary to detect objects at an earlier stage.
[0012] The present disclosure has been made in consideration of the above circumstances, and aims to determine the object detection rate by operators at a monitoring center.
[0013] In order to achieve the above object, the present disclosure provides an object detection rate calculation device that calculates an object detection rate indicating the proportion of observers who were able to actually detect an object when multiple observers viewed surveillance video, based on current and past video quality parameters acquired from a camera unit provided in a mobile body that is capable of autonomous movement, and distance information indicating the distance from the mobile body or the camera unit to a predetermined object.
[0014] As described above, the present disclosure provides the effect of being able to determine the object detection rate by an operator at a monitoring center.
[0015] FIG. 1 is an overall configuration diagram of a communication system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a model showing the relationship between an object detection rate and an assumed object distance relative to a video bit rate. FIG. 3 is a diagram illustrating an example of a model showing the relationship between an object detection rate and an assumed object distance relative to the speed of a moving body. FIG. 4 is a diagram illustrating input information when the bit rate is fixed and the speed is variable. FIG. 5 is a diagram illustrating input information when the bit rate is variable and the speed is fixed. FIG. 6 is a diagram illustrating input information when the bit rate is variable and the speed is also variable. FIG. 7 is an electrical hardware configuration diagram of each device and terminal according to an embodiment.
[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0017] First Embodiment A first embodiment will be described with reference to FIGS. 1 to 9. FIG.
[0018] [System Configuration of First Embodiment] The configuration of a communication system 1a according to the first embodiment will be outlined below with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to the first embodiment. Note that the values shown in Fig. 1 (10 Mbps, 5 Mbps, 30 km / h, 35 km / h, 20 m, 70%) are merely examples and are not intended to be limiting.
[0019] As shown in FIG. 1, the communication system 1a is constructed by various devices (a photographing device 3, a speed detection device 4, an object detection rate calculation device 5, a setting device 6, and a communication device 7) mounted on a mobile object A that is capable of moving automatically, such as a vehicle, and an operator terminal 8 placed on the remote monitoring center B side.
[0020] Basically, moving body A operates automatically, but by displaying images from multiple moving bodies on the operator terminal 8 of the monitoring center B as shown in Fig. 9, the operator of the monitoring center B can monitor the movement status of each moving body and remotely operate a specific moving body depending on the movement status. As a result, when the operator is viewing the image from a specific moving body, if an object or person that may interfere with driving suddenly appears, the operator can remotely avoid an accident.
[0021] There may be a plurality of each of the photographing device 3, speed detection device 4, object detection rate calculation device 5, setting device 6, communication device 7, and operator terminal 8. In particular, there may be four or more photographing devices 3 to photograph the front, right side, left side, and rear of the moving object A.
[0022] In this embodiment, the mobile object A is assumed to be, for example, a vehicle (automobile) having a communication function and an automatic driving function. However, the mobile object A is not limited to an automobile, and may be an agricultural machine such as a tractor having a communication function and an automatic driving function. Furthermore, the mobile object A may be a ship, an aircraft, or the like, other than a vehicle.
[0023] Of these, the image capturing device 3 has a camera mounted on the moving body A and an encoder for compressing video data, encodes the video data obtained by capturing images of the outside of the moving body A, and transmits the encoded video data to the communication device 7. Furthermore, the image capturing device 3 has a bit rate extraction function, extracts the video bit rate from the encoded video data, and transmits bit rate information indicating the bit rate per unit time t (for example, 1 second) to the object detection rate calculation device 5. Note that the image capturing device 3 and the encoder (device) may be separate entities.
[0024] The speed detection device 4 detects the speed (e.g., h per hour) of the moving object A, and transmits speed information indicating the detected speed to the object detection rate calculation device 5. This speed information is also transmitted every unit time t, similar to the bit rate information.
[0025] The object detection rate calculation device 5 includes a calculation unit (first calculation unit) that calculates the object detection rate based on current and past bit rate information acquired from the image capture device 3, speed information acquired from the speed detection device 4, and distance information acquired from the setting device 6. This calculation unit is a function realized by executing a program. The object detection rate indicates the percentage of observers who actually detect (recognize) an object when viewing surveillance video, and can also be referred to as the object recognition rate. For example, if there are 100 observers with various visual acuity levels, and 50 of them are able to detect an object captured in the surveillance video, the object detection rate is 50%. The object detection rate calculation device 5 then transmits current object detection rate information indicating the calculated object detection rate to the operator terminal 8 of the monitoring center B via the communication device 7. As a result, the operator terminal 8 displays the object detection rate information (e.g., 80%) for each moving object along with the video (monitoring video) from each moving object, as shown in FIG. 9 (described later). The method for calculating the object detection rate will be described in detail later.
[0026] The setting device 6 is a device that sets distance information used to calculate the object detection rate by the object detection rate calculation device 5. The distance information indicates the distance between the moving body A (or the image capture device 3) and the object, and is a predetermined fixed value such as 10 m, 20 m, or 30 m.
[0027] The communication device 7 can communicate data with the operator terminal 8 via a communication network 100. The communication network 100 is constructed using a mobile network, the Internet, etc. The Internet also includes a space Internet that passes through outer space using an artificial satellite or the like.
[0028] The operator terminal 8 is a terminal such as a PC (Personal Computer) used by an operator at the monitoring center B. The operator terminal 8 has a decoder for decoding encoded video data.
[0029] 2 to 6, a method for calculating the object detection rate in the object detection rate calculation device 5 will be described. Here, a method for calculating the object detection rate dr using the bit rate br of the video input to the device, the speed v, and the distance d between the moving body A (or the image capture device 3) and the object will be described.
[0030] <Relationship between object detection rate and object distance relative to bit rate> Video quality is correlated with bit rate, and the higher the bit rate, the higher the video quality. The corresponding relationship between object distance and object detection rate relative to bit rate is shown in Figure 2. Figure 2 is a diagram showing an example of a model (first object detection rate model) showing the relationship between the object detection rate relative to the video bit rate and the assumed object distance.
[0031] The higher the bit rate, the higher the object detection rate for the same object distance. On the other hand, once the bit rate reaches a certain level, the video quality reaches almost its upper limit, and the object detection rate remains unchanged even at bit rates above that level. Therefore, the object detection rate calculation device 5 calculates the object detection rate using an object detection rate model in which the object detection rate increases as the bit rate increases, but the object detection rate remains almost unchanged when the bit rate is above a predetermined value (first predetermined value). Furthermore, the object detection rate calculation device 5 calculates the object detection rate using an object detection rate model in which the object detection rate increases when the distance between the moving body A and the object is below a predetermined value (second predetermined value) (relatively close), and the object detection rate remains above a certain value even when the bit rate is below a predetermined value (third predetermined value) (relatively low). On the other hand, the object detection rate calculation device 5 calculates the object detection rate using a model (first object detection rate model) of object detection rate in which the object detection rate decreases as the distance between the moving body A and the object increases, and the object detection rate becomes 0 when the distance is greater than a predetermined value (fourth predetermined value) (too far).
[0032] The object detection rate calculation device 5 uses the first object detection rate model to calculate the object detection rate based on the current and past parameters and the above-mentioned distance.
[0033] <Relationship between Object Detection Rate and Expected Object Distance with Respect to Speed> Furthermore, by inputting the speed of moving body A, different detection rates can be output when the speed of moving body A is different. FIG. 3 is a diagram showing an example of a model (second object detection rate model) illustrating the relationship between the object detection rate and the expected object distance with respect to the speed of the moving body. FIG. 3 illustrates the relationship between the object detection rate and the object distance when the speed of moving body A is 40 km / h and 10 km / h. When moving body A is moving at a high speed, the movement of the background and objects in the video becomes larger (faster), making video encoding more difficult. Therefore, even when video is delivered at the same bit rate, it is believed that the video quality and the object detection rate will decrease. Therefore, the object detection rate calculation device 5 calculates the object detection rate using this property. Furthermore, when moving body A is moving at a high speed, even if the distance to the object when it appears in the video is the same, the distance moving body A travels will be longer before a human, such as an operator, detects the object. Therefore, the assumed object distance in the object detection rate is considered to be short, and the object detection rate calculation device 5 calculates the object detection rate using this property.
[0034] The object detection rate calculation device 5 has a calculation unit (second calculation unit) that calculates the object detection rate based on the current and past speeds and the above-mentioned distances using a second object detection rate model. This calculation unit is a function realized by executing a program.
[0035] The object detection rate calculation device 5 may also have a calculation unit (third calculation unit) that uses the first object detection rate model and the second object detection rate model to calculate the object detection rate based on the current and past parameters, the current and past velocities, and the distance described above. This calculation unit is a function realized by executing a program.
[0036] 4 to 6, a method for calculating an object detection rate by the object detection rate calculation device 5 will be described. Note that a method for calculating a detection rate when bit rate information and speed information are actually input to the object detection rate calculation device 5 will be described in detail.
[0037] Here, an equation for the object detection rate calculated using the current bit rate, current speed, and current distance to the object is used as a reference, and this is used to calculate the object detection rate when at least one of the bit rate and speed changes. First, the reference object detection rate is calculated using the current bit rate (br), speed (v), and distance to the object (d). If the equation for calculating the object detection rate is expressed as the equation f(br, v, d) with bit rate information, speed information, and distance information as input, the object detection rate dr can be expressed as the following (Equation 1) using the function f.
[0038] As shown in Figures 2 and 3, the function f(br, v, d) has the characteristic that the object detection rate, which is its output, increases as the distance d decreases, and further has the above-mentioned properties regarding the bit rate and speed. With this in mind, f(br, v, d) is calculated using one of the following formulas. Note that f(br, v, d) is defined in the range from 0 to 1, and in each of the formulas below, when f(br, v, d) is 0 or less, the object detection rate is defined as 0, and when f(br, v, d) is 1 or more, the object detection rate is defined as 1. Note that the formula used to calculate the object detection rate is not limited to the one shown below.
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046] Note that x0 and A in (Equation 9) are expressed as follows:
[0047] x0 = b1×log(b2 + br) + b3 - (b 4 + b5×log(b6 + br))×v A = -c1×log(c2 + br) +c3 +(c4 - c5×log(c6 + br))×v Alternatively, A can be expressed as follows:
[0048] A = -c1 × exp(-(c2 × x0) - c3) + c4 Here, in (Equation 2) to (Equation 9), a1 - a4, b1 - b6, c1 - c6 are coefficients that differ for each equation. Also, e represents the base of the natural logarithm. Here, by setting the above coefficients to values that correspond to conditions such as the size of the object, the size of the display, the size of the object reflected on the display, and the weather, it is possible to estimate the object according to the driving conditions and monitoring environment.
[0049] <Past Bit Rate and Speed> Next, we will explain how the object detection rate calculation device 5 inputs video bit rate information and speed information of moving body A in chronological order at each of the times t seconds before the current time, 2t seconds before the current time, etc., where the current time is T, and calculates the current object detection rate using this information.
[0050] (Bit rate is fixed and speed is variable) Fig. 4 is a diagram showing input information when the bit rate is fixed and speed is variable. In Fig. 4, as an example, the bit rate br x The bit rate is fixed at 5Mbps, but the speed fluctuates. As shown in Figure 4, the current bit rate is br0, the current speed is v0, and the bit rate t seconds ago was br t (=br0), the velocity t seconds ago was v t , the bit rate 2t seconds ago was br 2t (=br0), the velocity 2t seconds ago was v 2t , .... It is also assumed that bit rate information and speed information are input to the object detection rate calculation device 5 at each time.
[0051] Here, since the speed differs at each time, the distance of moving object A from the object at a past time is calculated based on the speed at each time. The distance that moving object A traveled from nt seconds ago to the current time is calculated as the corrected distance dc nt Then, the correction distance is calculated using, for example, the following (Equation 10).
[0052] The object detection rate calculation device 5 calculates the object detection rate nt seconds ago (n = 1, 2, ...) using this (Equation 10). However, at past times, the position of moving object A at the current time is different from the current position, and moving object A is located closer to the current position, so the distance to be detected must be corrected according to the speed. At time T-nt, which is nt seconds before the current time T, the distance to be corrected from the current distance is calculated as dc nt Then, the object detection rate f nt is expressed as the following (Equation 11).
[0053] The current object detection rate is calculated by taking into account the current and past object detection rates. When bit rate information up to Nt seconds ago is input, the object detection rate dr is expressed by the following (Equation 12). Note that although N = 3 in Figure 4, the value of N is not limited to N = 3.
[0054] (When the bit rate is variable and the speed is fixed) Fig. 5 is a diagram showing input information when the bit rate is variable and the speed is fixed. In Fig. 5, as an example, the bit rate br x The current bit rate is br0, the current speed is v0, and the bit rate t seconds ago is br t , the velocity t seconds ago was v t (= v0), the bit rate 2t seconds ago was br 2t , the velocity 2t seconds ago was v 2t (=v0), .... It is also assumed that bit rate information and speed information are input to the object detection rate calculation device 5 at each time.
[0055] Here, if bit rate fluctuations are not taken into consideration, the object detection rate can be expressed as f(br0, v0, d) based on the bit rate and speed at the current time T. On the other hand, t seconds ago, video was played at a bit rate of 10 Mbps, which is higher than the current bit rate of 5 Mbps. In this case, because the object detection rate for an object d [m] ahead at the current time T is below a predetermined value, even if the operator has difficulty detecting the object, there is a possibility that the object was detected t seconds ago.
[0056] Therefore, to find the possibility that the same object was detected at a past time, the object detection rate calculation device 5 calculates the object detection rate nt seconds ago (n = 1, 2, ...). However, at a past time, the position of moving object A at the current time is different from that at the current time, and moving object A is located closer to the current time. Therefore, in this case too, the distance to be detected must be corrected according to the speed. At time T-nt, which is nt seconds before the current time T, the distance to be corrected from the current distance is calculated as dc nt Then, the object detection rate f nt is expressed as the following (Equation 13).
[0057] Here, the correction distance dc nt When the speed is constant, is expressed as the product of time and speed (here, speed per hour), and therefore the object detection rate calculation device 5 calculates using the following (Equation 14).
[0058] The current (present) object detection rate is calculated taking into account the current and past object detection rates. When bit rate information up to Nt seconds ago is input, the object detection rate dr is expressed by the above (Equation 12). Note that although N=3 in FIG. 5, the value of N is not limited to N=3.
[0059] where b ktis a function that weights the current and past object detection rates, and is a value defined between 0 and 1. Furthermore, if the past object detection rate is high because the bit rate was high in the past, an operator at monitoring center B can remotely grasp the situation on-site even if the current object detection rate is low because the current bit rate is low. In this case, when detecting an object located d [m] ahead at the current time T, object detection rate calculation device 5 may select the largest of the past and current object detection rates. In that case, it can be expressed by the following (Equation 15).
[0060] (When both the bit rate and the speed are variable) Fig. 6 is a diagram showing input information when the bit rate and the speed are variable. In Fig. 6, as an example, the bit rate br x As shown in Figure 6, the current bit rate is br0, the current speed is v0, and the bit rate t seconds ago is br t , the velocity t seconds ago was v t , the bit rate 2t seconds ago was br 2t , the velocity 2t seconds ago was v 2t , .... Assume that bit rate information and speed information are input to the object detection rate calculation device 5 at each time. In this case, similar to the case where the bit rate fluctuates in FIG. 5, the object detection rate is calculated taking into account the bit rate of past images. On the other hand, unlike the case where only the bit rate fluctuates, the speed also fluctuates, so similar to the case where the speed fluctuates in FIG. 4, the object detection rate calculation device 5 calculates the corrected distance dc using the past speed at each time. nt Therefore, the object detection rate calculation device 5 can calculate the object detection rate dr by combining the formulas used when only the speed fluctuates and when only the bit rate fluctuates. That is, when the corrected distance dc nt is expressed as (Equation 10), and the object detection rate f nt is shown in (Equation 13).
[0061] From these (Equation 10) and (Equation 13), the current object detection rate is expressed by, for example, (Equation 12) or (Equation 15).
[0062] where b kt is a function that weights the current and past object detection rates, and is a value defined between 0 and 1.
[0063] Note that if the object detection rate calculation device 5 calculates the object detection rate assuming that the speed of the moving object A is fixed (constant), the speed detection device 4 may be omitted. In this case, speed information is set in advance in the object detection rate calculation device 5. Also, if the object detection rate calculation device 5 calculates the object detection rate assuming that the video bit rate is fixed (constant), the image capture device 3 does not need to extract the bit rate. In this case, the bit rate is set in the object detection rate calculation device 5 in advance.
[0064] [Hardware Configuration] Next, the electrical hardware configuration of the object detection rate calculation device 5, the setting device 6, the communication device 7, and the operator terminal 8 will be described with reference to Fig. 7. Fig. 7 is a diagram illustrating the electrical hardware configuration of each device and terminal according to the embodiment.
[0065] As shown in FIG. 7, the object detection rate calculation device 5 is a computer that includes a CPU 101, a ROM 102, a RAM 103, an SSD 104, an external device connection I / F (Interface) 105, a network I / F 106, a display 107, an input device 108, a media I / F 109, and a bus line 110.
[0066] Of these, the CPU 101 controls the overall operation of the object detection rate calculation device 5. The ROM 102 stores programs such as IPL used to drive the CPU 101. The RAM 103 is used as a work area for the CPU 101.
[0067] The SSD 104 reads or writes various data under the control of the CPU 101. Note that instead of the SSD 104, a hard disk drive (HDD) may be used.
[0068] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB memory, and a printer.
[0069] The network I / F 106 is an interface for performing data communication via the communication network 100 .
[0070] The display 107 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays various images.
[0071] The input device 108 is a button, switch, keyboard, pointing device, etc., and is an example of an input means for accepting input operations such as characters, numbers, and various instructions.
[0072] The media I / F 109 controls reading and writing (storing) of data from and to a recording medium 109m such as a flash memory, etc. The recording medium 109m includes DVDs and Blu-ray Discs (registered trademarks).
[0073] The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101 shown in FIG.
[0074] The setting device 6, the communication device 7, and the operator terminal 8 have the same configuration as the object detection rate calculation device 5, and therefore a description thereof will be omitted.
[0075] [Processing or Operation of the Communication System According to the First Embodiment] Next, the processing or operation of each device in the communication system 1a will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is a flowchart showing a process for estimating and presenting an object detection rate according to the first embodiment. Fig. 9 is a diagram showing an example of a display screen on an operator terminal at a monitoring center.
[0076] S11: After the image capturing device 3 captures an image of the outside of the moving object A and obtains video data, the image capturing device 3 compresses the video data using an encoder and transmits it to the communication device 7. The communication device 7 then transmits the video data to the operator terminal 8 via the communication network 100.
[0077] S12 : The image capturing device 3 extracts the bit rate of the video using the encoding information of the video data, and transmits the bit rate information (for example, 5 Mbps, 10 Mbps) to the object detection rate calculation device 5 .
[0078] S13: The speed detection device 4 detects the current speed of the moving object A, and transmits speed information indicating this speed to the object detection rate calculation device 5.
[0079] S14: The object detection rate calculation device 5 calculates the current object detection rate (e.g., 70%) based on the bit rate information, speed information, and preset distance information (e.g., 20 m) using at least one of the models shown in Fig. 2 and Fig. 3. Then, the object detection rate calculation device 5 transmits object detection rate information indicating the current object detection rate to the communication device 7.
[0080] S15: The communication device 7 transmits the current object detection rate information to the operator terminal 8 via the communication network 100.
[0081] 9, the operator terminal 8 decodes the video data transmitted from each moving object in step S11, associates the video of each moving object with the object detection rate information transmitted from the communication device 7 in step S15, and displays the decoded video on a split screen. To associate the video with the object detection rate information, the video data and the object detection rate information may each contain identification information such as an ID. In this case, the image capture device 3 adds the identification information to the video data and bit rate information that it outputs. The object detection rate calculation device 5 then adds the identification information added to the bit rate information acquired from the image capture device 3 to the object detection rate information and outputs the information.
[0082] S16: If the movement of the moving object A has not ended (NO), the process returns to step S11. On the other hand, if the movement of the moving object A has ended (YES), the process shown in FIG.
[0083] [Modification of System Configuration of First Embodiment] Next, a communication system 1b according to a modification of the first embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing the overall configuration of a communication system according to a modification of the first embodiment. Note that the values shown in Fig. 10 (10 Mbps, etc.) are merely examples, similar to Fig. 1, and are not limited to these.
[0084] 1 in that speed information (e.g., 30 km / h) is set in advance by a setting device 6 and transmitted to an object detection rate calculation device 5, without using a speed detection device 4. In other words, this modification shows an example in which the speed of moving body A is fixed (constant).
[0085] Effects of First Embodiment and Modifications As described above, according to the first embodiment, it is possible to determine and present the object detection rate by the operator of the monitoring center B. As a result, the operator terminal 8 of the monitoring center B displays the monitoring video from each moving object in association with the object detection rate information for this monitoring video, as shown in Fig. 9, thereby eliminating the situation in which the operator does not notice that the video quality of the monitoring video has deteriorated.
[0086] Furthermore, even if the current object detection rate is below a predetermined value, the bit rate of past images and the past speed of moving body A can be tracked (considered) to calculate the current object detection rate, making it easier for the operator to recognize the object.
[0087] Second Embodiment Next, a second embodiment will be described with reference to FIGS.
[0088] [System Configuration of Second Embodiment] Next, a communication system 2a according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram showing the overall configuration of a communication system according to the second embodiment. Note that the values shown in Fig. 11 (10 Mbps, etc.) are merely examples, as in Fig. 1, and are not limited to these. Furthermore, the same devices or terminals as those in Fig. 1 are designated by the same reference numerals, and their description will be omitted.
[0089] In the communication system 2a of Fig. 11, the object detection rate calculation device 5 on the mobile object A side is installed on the monitoring center B side, and a communication device 9 is also installed in the monitoring center B. Video data encoded by the imaging device 3 is transmitted to an operator terminal 8 via a communication device 7. However, in this embodiment, it is the operator terminal 8, not the imaging device 3, that has the bit rate extraction function. Furthermore, the mobile object A does not have a setting device 6; instead, the distance is set in advance on the operator terminal 8 and transmitted to the object detection rate calculation device 5. Furthermore, speed information indicating the fluctuating speed of the mobile object A, detected by the speed detection device 4 of the mobile object A, is transmitted to the object detection rate calculation device 5 via the communication devices 7 and 9. This transmission is performed every unit time t (for example, 1 second).
[0090] As in the first embodiment, the object detection rate calculation device 5 calculates the current object detection rate based on the distance information, bit rate information, and speed information, and transmits it to the operator terminal 8 .
[0091] [Processing or Operation of Communication System According to Second Embodiment] Next, processing or operation of each device in the communication system 2a will be described with reference to Fig. 12. Fig. 12 is a flowchart showing processing for estimating and presenting an object detection rate according to the second embodiment.
[0092] S21: After the image capturing device 3 captures an image of the outside of the moving object A and obtains video data, the image capturing device 3 compresses the video data using an encoder and transmits the compressed video data to the communication device 7. The communication device 7 then transmits the video data to the operator terminal 8 via the communication network 100.
[0093] S22: The speed detection device 4 detects the current speed of the moving object A, and transmits speed information indicating this speed to the object detection rate calculation device 5.
[0094] S23: The communication device 7 transmits the video data to the operator terminal 8 and also transmits the speed information to the communication device 9.
[0095] S24: The communication device 9 transmits the speed information to the object detection rate calculation device 5.
[0096] S25: The operator terminal 8 extracts the video bit rate using the encoding information of the video data, and transmits the bit rate information (for example, 5 Mbps, 10 Mbps) to the object detection rate calculation device 5.
[0097] S26: The object detection rate calculation device 5 calculates the current object detection rate (e.g., 70%) based on the bit rate information, speed information, and preset distance information (e.g., 20 m) using at least one of the models shown in Fig. 2 and Fig. 3. Then, the object detection rate calculation device 5 transmits object detection rate information indicating the current object detection rate to the operator terminal 8.
[0098] As a result, as shown in FIG. 9 , the operator terminal 8 decodes the video data transmitted from each moving object in step S23, associates the object detection rate information for the video of each moving object transmitted from the object detection rate calculation device 5 in step S26, and displays the decoded video on a split screen. In this case, too, to associate the video with the object detection rate information, the video data and the object detection rate information may each contain identification information such as an ID. In this case, the image capture device 3 adds the identification information to the video data to be output. Then, when the operator terminal 8 extracts the bit rate, it adds the same identification information to the bit rate information. The object detection rate calculation device 5 adds the identification information added to the bit rate information acquired from the operator terminal 8 to the object detection rate information and transmits the object detection rate information to the operator terminal 8.
[0099] S27: If the movement of the moving object A has not ended (NO), the process returns to step S21. On the other hand, if the movement of the moving object A has ended (YES), the process shown in FIG.
[0100] [Modification of System Configuration of Second Embodiment] Next, a communication system 2b according to a modification of the second embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing the overall configuration of a communication system according to a modification of the second embodiment. Note that the values shown in Fig. 13 (10 Mbps, etc.) are merely examples, similar to Fig. 1, and are not limited to these.
[0101] 1 in that speed information (e.g., 30 km / h) is set in advance by a setting device 6 and transmitted to an object detection rate calculation device 5, without using a speed detection device 4. In other words, this modification shows an example in which the speed of moving body A is fixed (constant).
[0102] [Effects of the Second Embodiment and Modification] As described above, the second embodiment provides the same effects as the first embodiment. Furthermore, the second embodiment and its modification require fewer devices on the mobile unit A side compared to the first embodiment and its modification, which provides the effect of reducing the initial cost for receiving the remote monitoring and remote control services provided by the communication systems 2 a and 2 b.
[0103] [Supplementary Note] The present embodiment is not limited to the above embodiment, and various modifications and applications are possible, for example, as shown below.
[0104] (1) In the above embodiment, the image capture device 3 or the operator terminal 8 extracts the bit rate from the video data and transmits the bit rate information, and the object detection rate calculation device 5 calculates the object detection rate using the bit rate information. However, this is not limited to this. For example, the image capture device 3 or the operator terminal 8 may transmit resolution information indicating the resolution of the video or frame rate information indicating the frame rate of the video instead of the bit rate information, and the object detection rate calculation device 5 may calculate the object detection rate using the resolution information or frame rate information instead of the bit rate information. Note that the bit rate information, resolution information, and frame rate information are examples of parameters of video quality.
[0105] (2) Each device can be realized by a computer and a program, but this program can also be recorded on a (non-transitory) recording medium or provided via a communication network such as the Internet.
[0106] (3) The CPU 101 may be multiple, not just single.
[0107] [Relationship to the basic application] This patent application claims priority based on international patent application PCT / JP2023 / 019023, filed on May 22, 2023, and the entire contents of international patent application PCT / JP2023 / 019023 are incorporated herein by reference.
[0108] 1a, 1b, 2a, 2b Communication system A Mobile object B Monitoring center 3 Photographing device 4 Speed detection device 5 Object detection rate calculation device 6 Setting device 7 Communication device 8 Operator terminal 9 Communication device 100 Communication network
Claims
1. An object detection rate calculation device that calculates an object detection rate, which indicates the percentage of monitors who were able to actually detect an object when multiple monitors viewed surveillance footage, based on current and past image quality parameters obtained from a camera unit installed in a moving object, and distance information indicating the distance from the moving object or the camera unit to a specified object.
2. The object detection rate calculation device of claim 1, wherein the object detection rate calculation device uses a first object detection rate model that indicates the relationship between a parameter of image quality related to an image, the object detection rate, and a distance from the moving body or from an imaging device installed on the moving body to a specified object, and calculates the object detection rate based on the parameter and the distance.
3. An object detection rate calculation device that calculates an object detection rate, which indicates the percentage of monitors who were able to actually detect an object when multiple monitors viewed surveillance footage, based on the speed of a moving body obtained from a speed detection device installed on the moving body and the distance from the moving body to a specified object.
4. The object detection rate calculation device of claim 3, wherein the object detection rate calculation device uses a second object detection rate model that indicates the relationship between the object detection rate and the distance relative to the speed of the moving body, and calculates the object detection rate based on the speed and the distance.
5. An object detection rate calculation device that calculates an object detection rate indicating the percentage of observers who were able to actually detect an object when multiple observers viewed surveillance footage, based on image quality parameters related to images captured by an imaging device installed on a moving body, the speed of the moving body acquired from a speed detection device installed on the moving body, and the distance from the moving body or the imaging device to a specified object.
6. The object detection rate calculation device of claim 5, wherein the object detection rate calculation device uses a first object detection rate model showing the relationship between the object detection rate and the distance for the parameters, and a second object detection rate model showing the relationship between the object detection rate and the distance for the speed of the moving body, and calculates the object detection rate based on the parameters, the speed and the distance.
7. An object detection rate calculation device that calculates an object detection rate indicating the percentage of observers who were able to actually detect an object when multiple observers watch surveillance footage, based on current and past image quality parameters related to images captured by an imaging device installed in a moving body, the current and past speed of the moving body obtained from a speed detection device installed in the moving body, and the distance from the moving body or the imaging device to a specified object.
8. An object detection rate calculation method in which a computer calculates an object detection rate indicating the percentage of observers who were able to actually detect an object when multiple observers viewed surveillance footage, based on current and past image quality parameters obtained from a camera unit installed in a moving object, and distance information indicating the distance from the moving object or the camera unit to a specified object.
9. A program for causing a computer to execute the method according to claim 8.
Citation Information
Patent Citations
Image processing apparatus
JP2010003254A
Image recognition evaluation system and image recognition evaluation apparatus
JP2016126436A
Image generation program, image generation device, and image generation method
JP2022054664A
Visible distance estimation using deep learning in autonomous machine application
JP2023051713A
Object recognition device
JP2023062538A
Cited By
Object detection rate calculation device, object detection rate calculation method, and program
JP7786591B2
Object detection rate calculation device, object detection rate calculation method, and program
JPWO2024024776A1
