Evaluation device, evaluation method, and program

Abstract

To provide an evaluation device capable of comprehensively evaluating an observable space and a dead space in a monitoring space.SOLUTION: An evaluation device includes a depth information acquisition unit that acquires depth information indicating a distance between observation means for observing a monitored space and an object present in the monitored space, a posture information acquisition unit that acquires posture information including a position and a posture of the observation means at a time when the depth information is acquired, a parameter acquisition unit that acquires a characteristic parameter of the observation means, and a calculation processing unit that generates voxel data in which attribute information indicating whether each of a plurality of voxels obtained by dividing the monitored space is observable from the observation means or a blind spot is registered on the basis of the depth information, the posture information, and the characteristic parameter.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an evaluation device, an evaluation method, and a program. [Background technology]

[0002] When monitoring a space, sensors such as cameras, LiDAR, and radar, as well as the naked eye, are generally used as observation means. The observation coverage areas (field of view width, observation limit distance, etc.) of these observation means have been created and displayed in three-dimensional space, and the best placement of the observation means has been determined and studied. For example, Patent Document 1 describes a technology for calculating the visible area (visible space) within a monitored space according to the placement of surveillance cameras. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-010069 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional methods of determining and examining placement based on the inherent observation coverage of each observation means make it difficult to grasp and evaluate spaces in the monitored space that are obscured by objects (blind spots). Furthermore, conventional technologies evaluate whether an observation target can be detected in a monitored space by setting the positional relationship between the observation means and the observation target in advance and simulating whether an object is present between them. However, this evaluation method makes it difficult to comprehensively grasp and evaluate all spaces with blind spots. This can result in the risk of missing evaluations of observation targets (threats) approaching from unexpected positions or trajectories. Furthermore, conventional technologies limit evaluations to specific observation means, such as surveillance cameras, making it necessary to conduct different evaluations depending on the observation means.

[0005] An object of the present disclosure is to provide an evaluation device, an evaluation method, and a program that can comprehensively evaluate observable spaces and blind spot spaces in a monitored space. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, the evaluation device includes a depth information acquisition unit that acquires depth information indicating the distance between an observation means that observes a monitored space and an object present in the monitored space, an attitude information acquisition unit that acquires attitude information including the position and attitude of the observation means at the time the depth information is acquired, a parameter acquisition unit that acquires characteristic parameters of the observation means, and a calculation processing unit that registers attribute information indicating whether each of a plurality of small spaces divided into the monitored space is observable from the observation means or a blind spot based on the depth information, the attitude information, and the characteristic parameters.

[0007] According to one aspect of the present disclosure, an evaluation method includes the steps of acquiring depth information indicating the distance between an observation means observing a monitored space and an object present within the monitored space, acquiring attitude information including the position and attitude of the observation means at the time the depth information is acquired, acquiring characteristic parameters of the observation means, and registering attribute information indicating whether each of a plurality of small spaces divided from the monitored space is observable from the observation means or a blind spot based on the depth information, the attitude information, and the characteristic parameters.

[0008] According to one aspect of the present disclosure, the program causes an evaluation device to perform the following steps: acquiring depth information indicating the distance between an observation means observing a monitored space and an object present within the monitored space; acquiring attitude information including the position and attitude of the observation means at the time the depth information is acquired; acquiring characteristic parameters of the observation means; and registering attribute information indicating whether each of a plurality of small spaces divided into the monitored space is observable from the observation means or is a blind spot based on the depth information, the attitude information, and the characteristic parameters. [Effects of the Invention]

[0009] According to the above aspect, it is possible to comprehensively evaluate observable spaces and blind spots in a monitored space. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing the overall configuration of an evaluation system according to a first embodiment. [Figure 2] 1 is a block diagram showing a functional configuration of an evaluation device according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of processing performed by the evaluation device according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of voxel data according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the functional configuration of an evaluation device according to a second embodiment. [Figure 6] 10 is a flowchart showing an example of processing performed by an evaluation device according to a second embodiment. [Figure 7] FIG. 11 is a diagram showing an example of voxel data according to the third embodiment. [Figure 8] FIG. 13 is a diagram showing an example of voxel data according to the fourth embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a tree structure of voxel data according to the fourth embodiment. [Figure 10] FIG. 13 is a diagram showing an example of voxel data according to the fifth embodiment. [Figure 11] FIG. 1 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] First Embodiment The first embodiment will be described in detail below with reference to FIGS.

[0012] (Overall configuration of the evaluation system) 1 is a schematic diagram showing the overall configuration of an evaluation system according to the first embodiment. The evaluation system 1 includes an evaluation device 10 and an observation means 20 that observes a monitored space 3.

[0013] The evaluation device 10 evaluates the spaces that can be observed by the observation means 20 in the monitored space 3 and the spaces that cannot be observed (blind spots) due to the influence of objects 31 present in the monitored space 3 or the attitude of the observation means 20. In this embodiment, an example will be described in which the evaluation device 10 performs evaluation based on actual observation data from the observation means 20 installed in the monitored space 3.

[0014] The observation means 20 observes a monitored space 3 set in any area, such as outdoors or inside a facility, or in any area around a moving object (such as a vehicle or a drone for facility inspection), and acquires observation data that can detect whether a threat exists in the monitored space 3. Threats include, for example, intrusions into the monitored space (such as people, animals, or other moving objects), malfunctions in equipment, fires, and disasters that occur in the monitored space, and various other threats. The observation means 20 is a sensor such as a camera, LiDAR, or radar. For example, if the monitoring target is the presence or behavior of people, the observation means 20 may be a general camera that captures still images or videos. Furthermore, if the monitoring target is the temperature of plant equipment, the observation means 20 may be an infrared camera. In this way, the observation means 20 may be arbitrarily changed depending on the monitoring target. The observation means 20 is provided with a depth sensor 21 that can acquire depth information (distance information) only when the evaluation device 10 performs evaluation. If the observation means 20 is a device that can acquire depth information, such as a depth camera or LiDAR, the depth sensor 21 may be omitted.

[0015] (Functional configuration of evaluation device) 2 is a block diagram showing the functional configuration of the evaluation device according to the first embodiment. As shown in FIG. 2, the evaluation device 10 includes a processor 11, a memory 12, a storage 13, and a communication interface 14.

[0016] The processor 11 operates according to a predetermined program to function as a depth information acquisition unit 111, an orientation information acquisition unit 112, a parameter acquisition unit 113, a calculation processing unit 114, and an output unit 115.

[0017] The depth information acquisition unit 111 acquires depth information indicating the distance between the observation means 20 and an object 31 present in the monitored space 3. In this embodiment, the depth information acquisition unit 111 acquires the depth information from the depth sensor 21.

[0018] The orientation information acquisition unit 112 acquires orientation information including the position and orientation of the observation means 20 at the time the depth information is acquired. The orientation information is a parameter obtained by connecting a rotation matrix and a translation matrix in the horizontal direction.

[0019] The parameter acquisition unit 113 acquires characteristic parameters of the observation means 20. The characteristic parameters are internal parameters of the sensor. For example, if the observation means 20 is a camera, the characteristic parameters include the focal length, the position of the optical axis, and the like.

[0020] The calculation processing unit 114 registers attribute information indicating whether each of a plurality of small spaces obtained by dividing the monitored space 3 is observable from the observation means 20 or a blind spot based on depth information, attitude information, and characteristic parameters. The small spaces are, for example, the monitored space 3 divided by a square lattice (so-called voxels). In this embodiment, an example will be described in which the small spaces are voxels and the calculation processing unit 114 generates small space data (voxel data) in which attribute information is registered in the voxels.

[0021] The output unit 115 outputs (displays) the voxel data on a display device (not shown) connected to the evaluation device 10. The output unit 115 may also output (transmit) the voxel data to a terminal device of an administrator or operator of the observation means 20 via the communication interface 14.

[0022] The memory 12 has a memory area necessary for the operation of the processor 11 .

[0023] The storage 13 is a so-called auxiliary storage device, such as a hard disk drive (HDD), a solid state drive (SSD), etc. The storage 13 stores data that each part of the processor 11 acquires, generates, and refers to during processing.

[0024] The communication interface 14 is an interface for transmitting and receiving various data to and from external devices. In this embodiment, the communication interface 14 receives observation data and the like from the observation means 20 and the depth sensor 21. The received data is stored in the storage 13.

[0025] (Example of processing by evaluation device) 3 is a flowchart showing an example of processing by the evaluation device according to the first embodiment. The flow of processing by the evaluation device 10 will be described with reference to FIG.

[0026] First, the evaluation device 10 acquires input information (step S101). The input information includes depth information, orientation information, and characteristic parameters.

[0027] Specifically, the depth information acquisition unit 111 acquires depth information measured by a depth sensor 21 (or the observation means 20 itself) provided in the observation means 20. Here, an example will be described in which the observation means 20 is a depth camera, and the depth information is acquired directly from the observation means 20 instead of the depth sensor 21. The depth information is a mapping (depth map) of information indicating the depth (distance) from the observation means 20 to an object included in each pixel, associated with each pixel.

[0028] The attitude information acquisition unit 112 acquires attitude information of the observation means 20 at each time that depth information (depth map) is acquired. The observation means 20 may capture images while changing its attitude at a fixed position, or may be mounted on a moving body such as a UAV and capture images while changing both its position and attitude. In this way, when the position and attitude of the observation means 20 change from moment to moment, the attitude information acquisition unit 112 acquires multiple pieces of attitude information corresponding to the depth information at each time. Note that when the position and attitude of the observation means 20 are fixed, only one set of depth information and attitude information acquired at a certain time may be used.

[0029] The parameter acquisition unit 113 acquires characteristic parameters of the observation means 20. The characteristic parameters of the observation means 20 (focal length, optical axis position, etc.) are fixed values ​​set, for example, before the start of imaging. The parameter acquisition unit 113 may store the characteristic parameters of each observation means in the storage 13 in advance, and acquire the characteristic parameters of the observation means 20 to be evaluated from the storage 13.

[0030] Next, the calculation processing unit 114 evaluates the observable or blind spots in the monitored space 3 of the observation means 20 based on the input information. The input information includes depth information and orientation information corresponding to each of M frames (or M still images) included in the video captured by the observation means 20. The calculation processing unit 114 sequentially selects the ith frame (i=1, 2, ..., M) from the M frames and sets it as the processing frame (step S102).

[0031] The calculation processing unit 114 also allocates N voxels within the observation field of view of the observation means 20 (step S103), and evaluates whether each voxel has been observed. For example, the calculation processing unit 114 sequentially selects the jth (j=1, 2, ..., N) voxel from the N voxels, and evaluates the voxel based on the depth information (step S104).

[0032] FIG. 4 is a diagram showing an example of voxel data according to the first embodiment. FIG. 4 illustrates a part of the voxel space. For example, as shown in FIG. 4, the calculation processing unit 114 registers attribute information indicating any one of "unobserved", "near the surface" of the object 31, and "observed (observable)" in each voxel. In the example of FIG. 4, the values of the attribute information indicating "unobserved", "near the surface", and "observed" are negative values ("-1"), zero, and positive values ("1"), respectively. The calculation processing unit 114 initializes each voxel with attribute information indicating "unobserved". When it is specified from the input information (depth information, pose information, characteristic parameters) that the voxel j corresponds to a voxel in the space where the surface of the object 31 is detected (a pixel with a depth value corresponding to within the visual field range), the attribute information indicating "near the surface" of the object 31 is registered. Similarly, when it is specified from the input information that the voxel j corresponds to a voxel in an empty space where the object 31 does not exist, the attribute information indicating "observed (observable)" is registered. For example, voxels corresponding to the space in front of the object 31 (the space from the observation means 20 to the depth value indicating the surface of the object 31), the space where the background is detected (a pixel with a depth value exceeding the visual field range), and the space in front of it become "observed" voxels. The other voxels remain at the initial value ("unobserved").

[0033] Next, the calculation processing unit 114 determines whether the evaluation of all voxels in the processing frame i has been completed (step S105). If the evaluation of all voxels has not been completed (j < N) (step S105; NO), the process returns to step S104 to evaluate the next voxel (j = j + 1). If the evaluation of all voxels has been completed (j = N) (step S105; YES), it is further determined whether the processing of all frames has been completed (step S106).

[0034] If the processing of all frames has not been completed (i < M) (step S106; NO), the process returns to step S102, and the next frame (i = i + 1) is set as the processing frame. If the processing of all frames has been completed (i = M) (step S106; YES), all voxel data within the specified range is extracted (step S107). For example, when the observation means 20 performs imaging while changing its position and orientation, there is a possibility that spaces outside the monitoring space 3 are included in the voxel space for some frames. In such a case, the calculation processing unit 114 extracts the voxel data included in the monitoring space 3 specified by the user in advance.

[0035] Next, the calculation processing unit 114 integrates the voxel data of each extracted frame, and sets the voxels with attribute information other than "observed" registered as blind spot voxels (step S108). For example, the calculation processing unit 114 registers attribute information indicating "blind spot" in voxels where the number of times "observed" has been registered is less than a predetermined number of times (for example, 1 time) throughout all frames. Also, in other embodiments, the calculation processing unit 114 may add the observation time of the voxel (the imaging time of the frame in which "observed" is registered) and the number of observation times in which "observed" is registered as attribute information.

[0036] Also, the output unit 115 outputs the voxel data (step S109). For example, the output unit 115 outputs and displays the voxel data on a display device (not shown) connected to the evaluation device 10.

[0037] In addition, the output unit 115 may accept a user's operation and extract blind spot voxels (voxels in which attribute information indicating a "blind spot" is registered) and output them in a visualized state (step S110). At this time, the output unit 115 may simultaneously output numerical data indicating the volume (volume, ratio, etc.) of the blind spot voxels relative to all voxels in the monitored space 3. By quantitatively evaluating the blind spot voxels, the user can evaluate and consider an operation plan for the observation means 20. For example, if the observation means 20 is a surveillance camera, the user can evaluate and verify an operation plan such as the optimal placement location of the observation means 20, whether or not to increase or decrease the number of observation means 20, and the placement pattern when multiple observation means 20 are installed. In addition, the user may use the voxel data and information on the blind spot voxels to understand the spatial usage status of the monitored space 3 and to evaluate and consider crime prevention measures (plans). Furthermore, if the observation means 20 is an on-board camera of an automobile, the user can grasp the blind spots of the on-board camera and evaluate and verify the optimal placement location, whether or not to increase or decrease the number of cameras, and the placement pattern, just like with a surveillance camera. Furthermore, if the observation means 20 is an equipment inspection camera mounted on a mobile object, the user may use it to consider the movement route of the mobile object (equipment inspection route) and to detect inspection omissions (blind spot voxels).

[0038] The output unit 115 may output voxels having attribute information specified by the user in a visualized state. For example, if the voxel attribute information includes the number of observations, voxels with a low number of observations may be extracted and visualized as voxels at risk of insufficient monitoring. Furthermore, if the voxel attribute information includes the observation time, for example, if the voxel data is old (a predetermined time or more has passed since the observation time), the observation data (image) of the observation means 20 corresponding to this voxel data may not be used for consideration, or the observation data may be reacquired.

[0039] 3, an example has been described in which the calculation processing unit 114 sets the initial value of a voxel to "unobserved," and voxels registered as "near the surface" and voxels that remain "unobserved" (initial value) are treated as blind spot voxels, but the present invention is not limited to this. In another embodiment, the calculation processing unit 114 may register attribute information indicating "unobserved (unobservable)" for voxels that are deeper (farther from the observation means 20) than voxels registered as "near the surface" based on the depth information.

[0040] 4 shows an example in which attribute information indicating "unobserved" is fixed to "-1" and attribute information indicating "observed" is fixed to "1", but this is not limiting. In another embodiment, the calculation processing unit 114 may register a value indicating the distance between each voxel and the surface of the object 31 as attribute information. In this case, the calculation processing unit 114 calculates the distance of a voxel closer to the surface of the object 31 as a positive value, and the distance of a voxel further back as a negative value. In other words, "unobserved" is represented by a negative distance, and "observed" is represented by a positive distance.

[0041] The method of evaluating the attributes of each small space obtained by dividing the monitored space 3 is not limited to the method of dividing it into voxels (= square lattice) described above. In other embodiments, there may be a method of dividing it into small spaces along polar coordinates and evaluating it, or a method of dividing it into a solid model of any shape and evaluating it. The solid model may have a shape that represents the visible area, for example, based on the field of view and depth information of the camera. Also, if two-dimensional evaluation is sufficient, it may be possible to divide a plane into pixels. In this case, the above-mentioned "voxel" is replaced with a term appropriate for each method. The same applies to the following embodiments.

[0042] (Action and effect) As described above, the evaluation device 10 according to this embodiment comprises a depth information acquisition unit 111 that acquires depth information indicating the distance between the observation means 20 that observes the monitored space 3 and an object present in the monitored space 3, an attitude information acquisition unit 112 that acquires attitude information including the position and attitude of the observation means 20 at the time the depth information is acquired, a parameter acquisition unit 113 that acquires characteristic parameters of the observation means 20, and a calculation processing unit 114 that generates voxel data for each of the multiple voxels into which the monitored space 3 is divided, with attribute information indicating whether the voxel is observable from the observation means 20 or is a blind spot based on the depth information, attitude information, and characteristic parameters.

[0043] In this way, the evaluation device 10 can comprehensively evaluate observable spaces and blind spot spaces in the monitored space 3. This allows the operation plan, such as the number and placement of the observation means 20, to be appropriately evaluated and verified without overlooking blind spot spaces.

[0044] Furthermore, based on the depth information, attitude information, and characteristic parameters, the calculation processing unit 114 registers attribute information indicating that the voxels are "observed (observable)" for voxels closer to the observation means 20 than the voxels corresponding to the space where the surface of the object 31 was detected, and for voxels corresponding to the space where the background of the monitored space 3 was observed and for voxels closer to the observation means 20 than the voxels, and registers attribute information indicating that the other voxels are "blind spots."

[0045] In this way, the evaluation device 10 can provide the user with voxel data that allows the user to easily and clearly grasp observable spaces and blind spot spaces.

[0046] In addition, the depth information acquisition unit 111 acquires depth information from the observation means 20 or from a depth sensor 21 installed in the same position and attitude as the observation means 20, and the attitude information acquisition unit 112 acquires attitude information including the actual position and attitude of the observation means 20 at the time the depth information is acquired.

[0047] In this way, the evaluation device 10 can evaluate the observation means 20 that are actually in operation. This allows the user to evaluate the operation plan, including the number and placement of the observation means 20 in operation, and verify whether any changes are necessary, based on the voxel data. Furthermore, for example, if the observation means 20 is a general camera, by adding a depth sensor 21 only during evaluation, there is no need to install and operate a large number of relatively expensive depth sensors 21 as observation means, and the system can be configured inexpensively.

[0048] The attribute information may also include the time of observation by the observation means 20 or the number of times it was observed.

[0049] By doing this, for example, voxels with a small number of observations can be visualized as voxels at risk of insufficient monitoring. Also, for example, when voxel data is old (a predetermined time or more has passed since the observation time), this can be used to decide not to use this voxel data for consideration or to regenerate the voxel data.

[0050] <Second embodiment> Next, a second embodiment will be described in detail with reference to Figures 5 and 6. Components common to the above-described embodiment will be assigned the same reference numerals, and detailed description will be omitted. In this embodiment, an example will be described in which the evaluation device 10 evaluates the space observable by the observation means 20 and the space in the blind spot based on simulated data generated by simulation, rather than on actual measurement data from the observation means 20.

[0051] (Functional configuration of evaluation device) 5 is a block diagram showing the functional configuration of an evaluation device according to the second embodiment. As shown in FIG.

[0052] The simulation unit 116 performs a simulation using an environmental model M1 including a three-dimensional model of the monitored space 3 and the object 31, and an observation means model M2 that simulates the observation means 20, and generates depth information observed by the observation means 20 in the monitored space 3.

[0053] The depth information acquisition unit 111 acquires the depth information generated by the simulation unit 116 .

[0054] The attitude information acquisition unit 112 acquires attitude information including the virtual position and virtual attitude of the observation means model M2 used when the simulation unit 116 generates the depth information.

[0055] (Example of processing by evaluation device) Fig. 6 is a flowchart showing an example of processing by the evaluation device according to the second embodiment. Here, processing by the evaluation device 10 to generate simulated observation data according to the virtual position and virtual attitude of the observation means 20 by simulation will be described with reference to Fig. 6.

[0056] First, the simulation unit 116 prepares input data (step S201). The input data is an environment model M1, which is a three-dimensional model of the monitored space 3 to be evaluated, and an observation means model M2, which simulates the observation means 20 used in the monitored space 3. As shown in the example of FIG. 5, multiple models M1 and M2 of each monitored space 3 and observation means 20 are pre-stored in the storage 13, and the user selects from these multiple models the model M1 or M2 that corresponds to the evaluation target. The user also specifies the length (number of frames) of the video to be generated in the simulation, the number of still images, etc. Then, the simulation unit 116 imports the input data into the simulation environment (step S202).

[0057] The simulation unit 116 also accepts the setting of simulation conditions by the user (step S203). The simulation conditions include, for example, a virtual position and a virtual orientation to be set in the observation means model M2. At this time, the user may directly input parameters of the virtual position and virtual orientation, or may intuitively specify the position and orientation using a GUI. The user may also set multiple combinations of virtual positions and virtual orientations so that, for example, the optimal arrangement of the observation means 20 can be evaluated.

[0058] Next, the simulation unit 116 executes a simulation based on the input data (models M1, M2) imported into the simulation environment and the simulation conditions, and generates simulated observation data that simulates the observation data observed in the monitored space 3 by the observation means 20 (step S203). The simulated observation data includes images captured at virtual positions and virtual attitudes and depth information (depth map).

[0059] When the simulation is completed, the depth information acquisition unit 111 acquires the depth information generated by the simulation unit 116 (step S205). Furthermore, the orientation information acquisition unit 112 acquires orientation information including the virtual position and virtual orientation of the observation means set when the simulation unit 116 generated this depth information (step S206). If the user directly inputs parameters in step S203, the orientation information acquisition unit 112 acquires these parameters as orientation information as is. Furthermore, if the user specifies the position and orientation using a GUI, the orientation information acquisition unit 112 converts the position and orientation indicated by the GUI into parameters (matrix) of the orientation information and acquires them. The parameter acquisition unit 113 acquires the characteristic parameters set in the observation means model M2 (step S207).

[0060] Furthermore, when the acquisition of the depth information, posture information, and characteristic parameters is completed, the simulation unit 116 determines whether the simulation of all frames is completed (step S208). If the simulation of the number of frames specified by the user is not completed (step S208; NO), the process returns to step S203 and the simulation of the next frame is performed.

[0061] On the other hand, if all simulations for the specified number of frames have been completed (step S208; YES), the process of generating simulated observation data is terminated. Thereafter, the evaluation device 10 executes the series of processes shown in FIG. 3 and evaluates the observation means 20 using the generated simulated observation data. That is, in step S101 of FIG. 3, the simulated observation data generated in FIG. 6 is acquired as input information instead of the actual measurement data of the observation means 20. The process from FIG. 3 onwards is the same as in the first embodiment.

[0062] (Action and effect) As described above, the evaluation device 10 according to this embodiment further includes a simulation unit 116 that performs a simulation using an environment model M1 including a three-dimensional model of the monitored space 3 and the object 31, and an observation means model M2 that simulates the observation means 20, and generates depth information observed by the observation means 20 in the monitored space 3. The depth information acquisition unit 111 acquires the depth information generated by the simulation unit 116, and the orientation information acquisition unit 112 acquires orientation information including the virtual position and virtual orientation of the observation means used when the simulation unit 116 generated the depth information.

[0063] In this way, the evaluation device 10 can simulate and comprehensively evaluate the observable space and the blind spot space according to the position and attitude of the observation means 20 before actually installing the observation means 20. This makes it possible to consider and create an appropriate operation plan, such as the number and placement of the observation means 20, that minimizes the blind spot space, for example.

[0064] <Third embodiment> Next, a third embodiment will be described in detail with reference to FIG. 7. Components common to the above-described embodiments are denoted by the same reference numerals, and detailed description thereof will be omitted. In this embodiment, an example will be described in which the evaluation device 10 registers attribute information that can distinguish between unobserved voxels inside an object 31 and unobserved voxels outside the object 31. This embodiment is applicable to both the first and second embodiments.

[0065] FIG. 7 is a diagram showing an example of voxel data according to the third embodiment. For example, the calculation processing unit 114 executes the following process in step S108 of FIG. 3. As shown in FIG. 7, the calculation processing unit 114 identifies the range in the voxel space where the object 31 exists based on the environmental model M1. Furthermore, for voxels in which attribute information of "unobserved" is registered (which remains the initial value), the calculation processing unit 114 sets different values ​​of the attribute information for voxels included inside the object 31 and voxels on the far side of the object 31 (the side farther from the observation means 20). In the example of FIG. 7, a negative value (e.g., "-2") indicating "unobserved (inside the object)" is registered as attribute information for voxels included inside the object 31, and a negative value (e.g., "-1") indicating "unobserved (outside the object)" is registered as attribute information for voxels on the far side of the object 31.

[0066] In this way, the evaluation device 10 can provide the user with information that enables the user to identify whether a space in a blind spot from the observation means 20 is inside the object 31 or outside (toward the back) of the object 31. For example, if the observation means 20 is a surveillance camera mounted on an automobile, there is a possibility that a threat to the automobile, such as another automobile or a person, is present in the space in the blind spot outside the object 31. In such a case, the output unit 115 of the evaluation device 10 may notify the user that the blind spot voxels outside the object 31 pose a risk. Furthermore, the output unit 115 may weight the blind spot voxels inside the object 31 differently from the blind spot voxels outside the object 31, evaluate the larger the amount (volume) of blind spot voxels outside the object 31, and evaluate the risk as being higher, and notify the user of the evaluation value.

[0067] <Fourth embodiment> Next, a fourth embodiment will be described in detail with reference to Figures 8 and 9. Components common to the above-described embodiments will be assigned the same reference numerals, and detailed description will be omitted. In this embodiment, an example will be described in which the evaluation device 10 varies the size of voxels.

[0068] FIG. 8 is a diagram showing an example of a tree structure of voxel data according to the fourth embodiment. FIG. 9 is a diagram showing an example of voxel data according to the fourth embodiment. Generally, the greater the number of voxels, the greater the processing load and data volume. Therefore, in this embodiment, the calculation processing unit 114 holds voxel data in a tree structure as in the example of FIG. 8, and reduces the data volume by aggregating layers with the same voxel value into the next higher layer. The voxel value indicates the distance from the surface of the object 31, with positive values ​​on the near side (the side closer to the observation means 20) of the surface of the object 31 and negative values ​​on the far side. As in the example of FIG. 9, the voxel values ​​change near the observation point of the object 31, so data is held in the finest voxels, which are the lowest layer of the data tree structure (layer n in FIG. 8). On the other hand, in the space away from the object 31, the voxel values ​​are equal, so the fine voxel data in the lower layer (layer n in Figure 8) with equal voxel values ​​is aggregated into the coarse voxel data in the layer one level above in the tree structure (layer n-1 in Figure 8) (by pruning the lower layer), thereby reducing the amount of data.

[0069] In this way, the evaluation device 10 reduces the voxel size (increases the number of voxels contained in a unit space) for voxels closer to the detected object 31, and increases the voxel size (decreases the number of voxels contained in a unit space) for voxels farther from the object 31. This makes it possible to reduce the amount of voxel data.

[0070] <Fifth embodiment> Next, a fifth embodiment will be described in detail with reference to Fig. 10. Components common to the above-described embodiments will be assigned the same reference numerals, and detailed description thereof will be omitted. In this embodiment, an example will be described in which attribute information of voxels is expanded to include observation data (physical quantities) from observation means 20.

[0071] FIG. 10 is a diagram illustrating an example of voxel data according to the fifth embodiment. For example, assume that the observation means 20 is an infrared camera capable of measuring the temperature of each part of the monitored space 3. In this case, the calculation processing unit 114 may register the temperature observed by the observation means 20 as attribute information for each voxel based on the observation data of the observation means 20, as in the example of FIG. 10. Furthermore, when outputting the voxel data, the output unit 115 may represent each voxel in a display mode (e.g., color) according to the temperature. In this way, when monitoring plant equipment, for example, the evaluation device 10 can provide the user with voxel data that allows the distribution of the surface temperature of the equipment to be understood. This allows the user to utilize the voxel data for purposes other than identifying blind spots, such as checking for the presence or absence of abnormalities based on the surface temperature of the equipment.

[0072] Similarly, it is assumed that the observation means 20 has a microphone capable of measuring sounds and the like within the monitored space 3. In this case, the calculation processing unit 114 may register the sound (sound pressure level) observed by the observation means 20 in the attribute information of each voxel based on the observation data of the observation means 20. Furthermore, when outputting the voxel data, the output unit 115 may express each voxel in a display mode (color, etc.) according to the sound pressure level. In this way, the evaluation device 10 can provide the user with voxel data that enables the presence or absence of noise (abnormal noise) from, for example, plant equipment, and the location of the sound source to be grasped.

[0073] Note that temperature and sound are examples of physical quantities to be included in the attribute information, and are not limited to these. In other embodiments, various physical quantities that can be measured by the observation means 20 (and various sensors included in the observation means 20) may be included as attribute information.

[0074] <Other embodiments> Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications are possible. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Furthermore, some processes may be executed in parallel.

[0075] <Computer configuration> 11 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 900 includes a processor 901, a main storage device 902, an auxiliary storage device 903, and an interface 904. The evaluation device 10 described above is implemented in the computer 900. The operations of the above-described processing units are stored in the auxiliary storage device 903 in the form of a program. The processor 901 reads the program from the auxiliary storage device 903, loads it into the main storage device 902, and executes the above-described processing in accordance with the program. The processor 901 also allocates a storage area in the main storage device 902 to be used in the above-described processing in accordance with the program.

[0076] The program may be for realizing some of the functions to be performed by the computer 900. For example, the program may be combined with other programs already stored in the auxiliary storage device 903 or other programs implemented in other devices to perform the functions. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0077] Examples of the auxiliary storage device 903 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. The auxiliary storage device 903 may be an internal medium directly connected to the bus of the computer 900, or an external medium (external storage device 910) connected to the computer 900 via the interface 904 or a communication line. Furthermore, when this program is distributed to the computer 900 via a communication line, the computer 900 that receives the program may load the program into the main storage device 902 and execute the above-described processing. In at least one embodiment, the auxiliary storage device 903 is a non-transitory tangible storage medium.

[0078] <Additional Notes> The above-described embodiment can be understood, for example, as follows.

[0079] (1) According to the first aspect, the evaluation device 10 includes a depth information acquisition unit 111 that acquires depth information indicating the distance between the observation means 20 that observes the monitored space 3 and an object 31 that exists within the monitored space 3, an attitude information acquisition unit 112 that acquires attitude information including the position and attitude of the observation means 20 at the time the depth information is acquired, a parameter acquisition unit 113 that acquires characteristic parameters of the observation means 20, and a calculation processing unit 114 that registers attribute information indicating whether each of the multiple small spaces into which the monitored space 3 is divided is observable from the observation means 20 or is a blind spot based on the depth information, attitude information, and characteristic parameters.

[0080] In this way, the evaluation device 10 can comprehensively evaluate observable spaces and blind spot spaces in the monitored space 3. This allows the operation plan, such as the number and placement of the observation means 20, to be appropriately evaluated and verified without overlooking blind spot spaces.

[0081] (2) According to the second aspect, in the evaluation device 10 according to the first aspect, the small space is divided by a square lattice.

[0082] In this way, the evaluation device 10 can easily and highly accurately evaluate the monitored space 3 that includes objects of various shapes.

[0083] (3) According to the third aspect, in the evaluation device 10 relating to the first or second aspect, the calculation processing unit 114, based on the depth information, attitude information, and characteristic parameters, registers attribute information indicating that the surface of the object 31 is observable in a small space closer to the observation means 20 than the small space corresponding to the space in which the surface of the object 31 was detected, and in a small space corresponding to the space in which the background of the monitored space 3 was observed and in a small space closer to the observation means 20 than the small space, and registers attribute information indicating that the other small spaces are blind spots.

[0084] In this way, the evaluation device 10 can provide the user with small space data that allows the user to easily and clearly grasp the observable space and the blind spot space.

[0085] (4) According to the fourth aspect, in the evaluation device 10 relating to any one of the first to third aspects, the depth information acquisition unit 111 acquires depth information from the observation means 20 or from a depth sensor 21 installed at the same position and attitude as the observation means 20, and the attitude information acquisition unit 112 acquires attitude information including the actual position and attitude of the observation means 20 at the time the depth information is acquired.

[0086] In this way, the evaluation device 10 can evaluate the observation means 20 that are actually in operation. This allows the user to evaluate the operation plan, including the number and placement of the observation means 20 in operation, based on the small space data, and verify whether any changes are necessary. Furthermore, for example, if the observation means 20 is a general camera, by adding a depth sensor 21 only during evaluation, there is no need to install and operate a large number of relatively expensive depth sensors 21 as observation means, and the system can be configured inexpensively.

[0087] (5) According to the fifth aspect, the evaluation device 10 relating to any one of the first to third aspects further includes a simulation unit 116 that performs a simulation using an environmental model M1 including a three-dimensional model of the monitored space 3 and the object 31 and an observation means model M2 that simulates the observation means 20, and generates depth information observed by the observation means 20 in the monitored space 3, and the depth information acquisition unit 111 acquires the depth information generated by the simulation unit 116, and the attitude information acquisition unit 112 acquires attitude information including the virtual position and virtual attitude of the observation means 20 used by the simulation unit 116 when generating the depth information.

[0088] In this way, the evaluation device 10 can simulate and comprehensively evaluate the observable space and the blind spot space according to the position and attitude of the observation means 20 before actually installing the observation means 20. This makes it possible to consider and create an appropriate operation plan, such as the number and placement of the observation means 20, that minimizes the blind spot space, for example.

[0089] (6) According to the sixth aspect, in the evaluation device 10 relating to the third aspect, the calculation processing unit 114 further registers attribute information indicating whether or not a small space located farther from the observation means 20 than the surface of the detected object 31 is contained inside the object 31 based on the environmental model M1 including a three-dimensional model of the monitored space 3 and the object 31.

[0090] By doing this, the evaluation device 10 can provide the user with information that enables the user to identify whether a space that is blind spot from the observation means 20 is inside the object 31 or outside (the far side of) the object 31.

[0091] (7) According to the seventh aspect, in the evaluation device 10 relating to any one of the first to sixth aspects, the calculation processing unit 114 further registers attribute information indicating the time at which the small space was observed by the observation means 20 or the number of times it was observed.

[0092] In this way, the evaluation device 10 can visualize, for example, small spaces that have been observed infrequently as small spaces at risk of insufficient monitoring. Also, for example, when small space data is old (a predetermined time or more has passed since the observation time), this can be used to determine whether to not use this small space data for consideration or to regenerate the small space data.

[0093] (8) According to the eighth aspect, in the evaluation device 10 according to any one of the first to seventh aspects, the calculation processing unit 114 further registers attribute information indicating physical quantities that can be measured by the observation means 20 in the small space.

[0094] In this way, the evaluation device 10 can provide small space data that can be used for purposes other than identifying blind spots, such as checking for the presence or absence of abnormalities based on the physical quantities measured in the monitored space 3.

[0095] (9) According to the ninth aspect, in the evaluation device 10 relating to any one of the first to eighth aspects, the calculation processing unit 114 reduces the size of the small space the closer it is to the detected object 31, and increases the size of the small space the farther it is from the detected object 31.

[0096] In this way, the evaluation device 10 can reduce the amount of small space data.

[0097] (10) According to the tenth aspect, the evaluation method includes the steps of acquiring depth information indicating the distance between the observation means 20 observing the monitored space 3 and an object 31 present in the monitored space 3, acquiring attitude information including the position and attitude of the observation means 20 at the time the depth information is acquired, acquiring characteristic parameters of the observation means 20, and generating small space data for each of a plurality of small spaces divided from the monitored space 3, which registers attribute information indicating whether the small space is observable from the observation means 20 or is a blind spot based on the depth information, attitude information, and characteristic parameters.

[0098] (11) According to the eleventh aspect, the program causes the evaluation device 10 to execute the following steps: acquiring depth information indicating the distance between the observation means 20 observing the monitored space 3 and an object 31 present in the monitored space 3; acquiring attitude information including the position and attitude of the observation means 20 at the time the depth information was acquired; acquiring characteristic parameters of the observation means 20; and generating small space data for each of a plurality of small spaces divided into the monitored space 3, which registers attribute information indicating whether the small space is observable from the observation means 20 or is a blind spot based on the depth information, attitude information, and characteristic parameters. [Explanation of symbols]

[0099] 1. Rating System 10 Evaluation equipment 11 processors 111 Depth information acquisition unit 112 Posture information acquisition unit 113 Parameter Acquisition Unit 114 Computational Processing Unit 115 Output section 116 Simulation Department 12 Memory 13. Storage 14 Communication Interface 20 Observation Methods 21 Depth sensor 3 Surveillance space 31 Object M1 Environmental Model M2 Observational Instrument Model

Claims

1. a depth information acquisition unit that acquires depth information indicating a distance between an observation means that observes a monitored space and an object that exists in the monitored space; an attitude information acquisition unit that acquires attitude information including the position and attitude of the observation means at the time when the depth information is acquired; a parameter acquisition unit that acquires characteristic parameters of the observation means; a calculation processing unit that registers attribute information indicating whether each of a plurality of small spaces obtained by dividing the monitored space is observable from the observation means or a blind spot based on the depth information, the attitude information, and the characteristic parameters; An evaluation device comprising:

2. The small spaces are divided into square lattices. The evaluation device according to claim 1 .

3. The calculation processing unit Based on the depth information, the pose information, and the characteristic parameters, registering attribute information indicating that the object is observable in a small space closer to the observation means than the small space in which the surface of the object is detected, and in a small space in which the background of the monitored space is observed and in a small space closer to the observation means than the small space in which the surface of the object is detected; Register attribute information indicating that the small space is a blind spot in other small spaces. The evaluation device according to claim 1 .

4. the depth information acquisition unit acquires the depth information from the observation means or from a depth sensor installed at the same position and orientation as the observation means; the attitude information acquisition unit acquires the attitude information including the actual position and attitude of the observation means at the time when the depth information is acquired. The evaluation device according to claim 1 or 2.

5. a simulation unit that performs a simulation using an environmental model including a three-dimensional model of the monitored space and the object, and an observation means model that simulates the observation means, and generates depth information observed by the observation means in the monitored space; the depth information acquisition unit acquires the depth information generated by the simulation unit, the attitude information acquisition unit acquires the attitude information including a virtual position and a virtual attitude of the observation means used when the simulation unit generates the depth information. The evaluation device according to claim 1 or 2.

6. the calculation processing unit further registers attribute information indicating whether a small space located farther from the observation means than the surface of the detected object is included inside the object based on an environmental model including a three-dimensional model of the monitored space and the object; The evaluation device according to claim 2 .

7. The calculation processing unit further registers attribute information indicating the time when the small space was observed by the observation means or the number of times the small space was observed. The evaluation device according to claim 1 or 2.

8. the calculation processing unit further registers attribute information indicating physical quantities measurable by the observation means in the small space; The evaluation device according to claim 1 or 2.

9. the calculation processing unit reduces the size of the small space as it is closer to the detected object, and increases the size of the small space as it is farther from the detected object; The evaluation device according to claim 1 or 2.

10. acquiring depth information indicating a distance between an observation means for observing a monitored space and an object present in the monitored space; acquiring attitude information including the position and attitude of the observation means at the time when the depth information was acquired; obtaining characteristic parameters of the observation means; registering attribute information for each of a plurality of small spaces obtained by dividing the monitored space, the attribute information indicating whether the small spaces are observable from the observation means or are blind spots based on the depth information, the attitude information, and the characteristic parameters; An evaluation method having the following characteristics.

11. acquiring depth information indicating a distance between an observation means for observing a monitored space and an object present in the monitored space; acquiring attitude information including the position and attitude of the observation means at the time when the depth information was acquired; obtaining characteristic parameters of the observation means; registering attribute information for each of a plurality of small spaces obtained by dividing the monitored space, the attribute information indicating whether the small spaces are observable from the observation means or are blind spots based on the depth information, the attitude information, and the characteristic parameters; A program that causes the evaluation device to execute the above.

Images