Video processing support device and video processing support method

Abstract

To enable appropriate processing of confidential information in video footage without requiring prior training. [Solution] The video processing support device 100 according to one aspect of the present invention comprises: a 3D reconstruction processing unit 1 that generates a 3D reconstruction model Mr composed of 3D point cloud information from video data D in which a space has been captured; a masking portion extraction unit 3 that extracts the area of ​​the object to be concealed in the 3D reconstruction model Mr based on a 3D CAD model Mc representing the space in which the object to be concealed is set, and the 3D reconstruction model Mr; and a masking processing unit 5 that applies masking processing to the area in the video data D corresponding to the area of ​​the object to be concealed identified by the area of ​​the object to be concealed.

Description

Technical Field

[0001] The present invention relates to a moving image processing support device and a moving image processing support method.

Background Art

[0002] When a captured moving image is provided to a third party, masking, mosaic processing or the like may be applied to the moving image. This is to conceal information that poses security risks, information that can identify an individual, or information that can identify the shooting location, which are captured in the moving image. Examples of information that poses security risks include facilities provided for nuclear protection at nuclear power plants and the like. Examples of information that can identify personal information include nameplates of private houses, license plates of automobiles and the like.

[0003] When performing concealment processing on a moving image, much more labor is required than when the processing target is a still image. This is because a moving image is generally composed of about 30 to 60 images (frames) per second, resulting in a large number of images to be processed. Therefore, even the work of checking whether information that must be concealed is captured targets all the images included in the moving image, which requires extremely much labor and time. In addition, when a moving image with frequently switched shooting angles of view or a moving image with a long shooting time is to be processed, even the work of finding the time zone in which the information to be concealed is captured may take more time than the shooting time.

[0004] In addition, when a moving image with frequently switched shooting angles of view is targeted, the time required for the moving image processing for concealment also increases. This is because when the shooting angle of view of the information to be concealed is frequently switched, it is necessary to change the position of the processing portion every time the angle of view is switched.

[0005] For example, Patent Document 1 discloses an image masking device comprising: an image input receiving unit that receives image input; a masking target area extraction unit that automatically extracts the masking target area from the input image; a masking application unit that applies masking to the extracted masking target area; and an image output unit that outputs the masked image. In the technology described in Patent Document 1, what appears to be a face image is extracted from the video using machine learning, so by using the technology described in Patent Document 1, the time from capturing the video to providing the video to a third party can be shortened. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2023-111974 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, the technology disclosed in Patent Document 1 requires that the learning model be pre-trained to recognize the appearance of the subject to be concealed. When the subject to be trained on the learning model is an image, it is generally considered necessary to prepare hundreds or more photographs of the subject for training. Furthermore, machine learning also requires tagging, a process called annotation, which also requires a significant amount of effort.

[0008] Furthermore, when using learned data to detect confidential objects, it is necessary in principle to tolerate some detection failures. However, tolerating detection failures can lead to situations where confidential processing is not performed on objects that should be confidential. Preventing such omissions in confidential processing requires manual double-checking, which in turn requires a significant amount of effort.

[0009] Furthermore, in nuclear protection facilities or equipment such as nuclear power plants, it can be difficult to distinguish between equipment for nuclear protection and equipment that is not, based on appearance alone. Therefore, unnecessary masking may be applied to subjects that do not need to be concealed. If unnecessary masking is applied, information that should be conveyed to third parties will be concealed.

[0010] This invention was made to solve the above-mentioned problems, and its purpose is to enable appropriate processing of confidential information in video without requiring prior training. [Means for solving the problem]

[0011] A video processing support device according to one aspect of the present invention comprises: a 3D reconstruction processing unit that generates a 3D reconstruction model composed of 3D point cloud information from video data in which a space has been captured; a confidential target area extraction unit that extracts the confidential target area in the 3D reconstruction model based on a 3D CAD model representing the space in which confidential targets are set, and the 3D reconstruction model; and a masking processing unit that applies masking processing to the area in the video data corresponding to the confidential target area identified by the confidential target area. [Effects of the Invention]

[0012] According to the present invention, it becomes possible to appropriately process confidential information in video without requiring prior training. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram showing an example of the functional configuration of a video image processing support device according to one embodiment of the present invention. [Figure 2] This figure shows an example of the configuration of a single frame included in video data relating to one embodiment of the present invention. [Figure 3] This figure shows an example configuration of a 3D reconstruction model generated by a 3D reconstruction processing unit according to one embodiment of the present invention. [Figure 4]This is a diagram showing a configuration example of a CAD model according to an embodiment of the present invention. [Figure 5] This is a diagram showing a state in which a three-dimensional reconstruction model and a CAD model according to an embodiment of the present invention are arranged in the same coordinate space. [Figure 6] This is a diagram showing an example of a point cloud extracted by a masking partial extraction unit according to an embodiment of the present invention. [Figure 7] This is a diagram showing a configuration example of a moving image point cloud correspondence table generated by a three-dimensional reconstruction processing unit according to an embodiment of the present invention. [Figure 8] This is a diagram showing an example of information indicated by the in-moving-image position item in the moving image point cloud correspondence table according to an embodiment of the present invention. [Figure 9] This is a diagram showing an example of information indicated by the point coordinate item in the moving image point cloud correspondence table according to an embodiment of the present invention. [Figure 10] This is a diagram showing an overview of moving image search processing by a moving image search unit according to an embodiment of the present invention. [Figure 11] This is a diagram showing a configuration example of a frame specified by a moving image time according to an embodiment of the present invention. [Figure 12] This is a diagram showing an example of a point cloud composed of points defined in a masking target moving image point cloud correspondence table according to an embodiment of the present invention. [Figure 13] This is a diagram showing an example of a frame that has undergone masking processing by a masking processing unit according to an embodiment of the present invention. [Figure 14] This is a flowchart showing an example of the procedure of moving image processing support processing by a moving image processing support apparatus according to an embodiment of the present invention. [Figure 15] This is a block diagram showing a hardware configuration example of a moving image processing support apparatus according to an embodiment of the present invention. [Mode for Carrying Out the Invention]

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the present specification and the drawings, elements having substantially the same functions or configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0015] <Configuration of Moving Image Processing Support Apparatus> First, the configuration of the moving image processing support apparatus according to the present embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing an example of the functional configuration of the moving image processing support apparatus 100. As shown in FIG. 1, the moving image processing support apparatus 100 includes a three-dimensional reconstruction processing unit 1, an alignment processing unit 2, a masking part extraction unit 3, a moving image search unit 4, a masking processing unit 5, and a moving image point cloud correspondence table T.

[0016] The three-dimensional reconstruction processing unit 1 generates three-dimensional stereoscopic information, that is, a three-dimensional point cloud, from input moving image data D. In the following description, the three-dimensional point cloud generated by the three-dimensional reconstruction processing unit 1 is referred to as a "three-dimensional reconstruction model".

[0017] The moving image data D is a moving image obtained by imaging a space with an imaging apparatus (not shown). The space refers to a certain room, work area, or the like, for example, a space including facilities, objects, and the like that should be kept confidential for nuclear protection reasons inside a nuclear power plant. Therefore, the moving image data D obtained by imaging such a space includes images of the facilities that should be kept confidential.

[0018] FIG. 2 is a diagram showing a configuration example of one certain frame included in the moving image data D. A frame F1 of the moving image data D shown in FIG. 2 shows a space including a window and a clock.

[0019] The three-dimensional reconstruction processing unit 1 generates a three-dimensional reconstruction model Mr (see FIG. 3) from the moving image data D using a technique such as SfM (Structure from Motion), for example. SfM is a method of reproducing a subject in captured images as three-dimensional point cloud data by analyzing a large number of photographs and moving images captured from various positions or directions.

[0020] Figure 3 shows an example of the configuration of the 3D reconstruction model Mr generated by the 3D reconstruction processing unit 1. Figure 3 shows the 3D reconstruction model Mr generated by the 3D reconstruction processing unit 1 based on the video data D, which includes frame F1 shown in Figure 2. In the 3D reconstruction model Mr shown in Figure 3, the space containing windows, clocks, etc., included in frame F1 shown in Figure 2 is reproduced using a 3D point cloud.

[0021] Furthermore, the 3D reconstruction processing unit 1 generates a video point cloud correspondence table T (see Figure 1) that indicates which pixel in the video data D corresponds to which time (frame) in the video data D, for each point constituting the 3D reconstruction model Mr. An overview of the process by which the 3D reconstruction processing unit 1 generates the video point cloud correspondence table T will be described later with reference to Figure 8, and an example of the configuration of the video point cloud correspondence table T will be described later with reference to Figure 7.

[0022] Returning to Figure 1, the explanation continues. The alignment processing unit 2 takes the 3D reconstruction model Mr and the CAD (Computer Aided Design) model Mc, both input from the 3D reconstruction processing unit 1, as input and aligns the two models. The CAD model Mc is a 3D CAD model that represents the same space as the space shown in the video data D, with information about the object to be masked pre-set.

[0023] Figure 4 shows an example of the configuration of the CAD model Mc. As shown in Figure 4, a masking area Am is defined in the CAD model Mc. In Figure 4, the masking area Am is indicated by a cloud mark with diagonal lines and is set to a position corresponding to the position of the clock in the 3D reconstruction model Mr shown in Figure 3.

[0024] The masking area Am may be specified by an area such as a bounding box, or by an object on the CAD model. When specifying an object as the masking area Am, the masking area Am may be set to be a certain extent larger than the object itself. Setting the masking area Am in this way prevents the object being masked from being inferred from its shape.

[0025] The alignment processing unit 2 aligns the 3D reconstruction model Mr with the CAD model Mc in which the masking region Am is set. Alignment is achieved by performing alignment of the X, Y, Z axes, yaw, pitch, and roll, as well as scaling, so that the corresponding positions in both models coincide within a predetermined error range.

[0026] For example, the alignment processing unit 2 aligns the two models by placing them in the same coordinate space so that the corners in the spatial location of the 3D reconstruction model Mr shown in Figure 3 coincide with the corresponding corners in the CAD model Mc. If the coordinate spaces of the two models are different, the alignment processing unit 2 uses a transformation matrix or the like to make the corresponding positions match.

[0027] Figure 5 shows the state in which the 3D reconstruction model Mr and the CAD model Mc are placed in the same coordinate space. In the example shown in Figure 5, the 3D reconstruction model Mr shown in Figure 3 and the CAD model Mc shown in Figure 4 are displayed superimposed on each other. The masking target region Am is placed superimposed on the position of the clock to be masked in the 3D reconstruction model Mr.

[0028] The 3D reconstructed model Mr and the CAD model Mc, which have undergone alignment processing by the alignment processing unit 2, are transmitted to the masking area extraction unit 3. The masking area extraction unit 3 (an example of a concealed area extraction unit) extracts the areas in the 3D reconstructed model Mr that overlap with the masked area Am (an example of a concealed area). Specifically, the masking area extraction unit 3 extracts the point cloud Pc (see Figure 6) of the portion of the 3D reconstructed model Mr shown in Figure 3 that overlaps with the masked area Am set in the CAD model Mc.

[0029] Figure 6 shows an example of the point cloud Pc extracted by the masking area extraction unit 3. As shown in Figure 6, the point cloud Pc is a point cloud located at a position corresponding to the masking target area Am, and constitutes the point cloud that includes the clock to be concealed.

[0030] Each point that makes up the point cloud Pc extracted by the masking area extraction unit 3 is assigned an identification number, and the masking area extraction unit 3 extracts the corresponding identification number for each point from the video point cloud correspondence table T (see Figure 7). The masking area extraction unit 3 then transmits the identification numbers of the extracted points to the video search unit 4.

[0031] The video search unit 4 extracts information from the video point cloud correspondence table T that corresponds to the identification numbers of each point in the point cloud Pc, based on the identification number information of each point in the point cloud Pc sent from the masking portion extraction unit 3, and generates a masking target video point cloud correspondence table Ta. The video search unit 4 then transmits the generated masking target video point cloud correspondence table Ta to the masking processing unit 5. An overview of the video search processing by the video search unit 4 and an example of the configuration of the masking target video point cloud correspondence table Ta will be described later with reference to Figure 10.

[0032] The masking processing unit 5 receives information about the point cloud Pc sent from the video search unit 4 and the video data D as input, performs masking on the video data D, and outputs the masked video data Dm. An overview of the masking process performed by the masking processing unit 5 will be described later with reference to Figures 11 to 13.

[0033] <Overview of 3D Reconstruction Process> Next, with reference to Figures 7 and 8, an overview of the 3D reconstruction process performed by the 3D reconstruction processing unit 1 will be described. Figure 7 shows an example of the configuration of the video point cloud correspondence table T generated by the 3D reconstruction processing unit 1. As shown in Figure 7, the video point cloud correspondence table T has the following items: "Point Number", "Video Time", "Position in Video", and "Point Coordinates". The "Point Number" is an identification number that identifies each point that makes up the 3D reconstruction model Mr. The "Video Time" indicates the timestamp in the video data D, and this video time identifies the frame in the video data D. The "Position in Video" item stores information about the coordinates of the point in the identified frame. The "Point Coordinates" item stores information about the coordinates of the point in the 3D reconstruction model Mr.

[0034] Figure 7 shows that the point associated with point number "1" is located at coordinates "a1,b1" in the frame where the video time is "00:00:02", and its coordinates in the 3D reconstruction model Mr are "x1,y1,z1".

[0035] Figure 8 shows an example of information indicated by the "location within the video" item in the video point cloud correspondence table T, and Figure 9 shows an example of information indicated by the "coordinates of a point" item in the video point cloud correspondence table T.

[0036] In a 2D image, the top left corner of frame F1 is considered the origin, and coordinates within frame F1 are represented accordingly. Figure 8 shows that the coordinates within frame F1 of point (pixel) P1, managed by point number "1", are "a1, b1", and the coordinates within frame F1 of point (pixel) P2, managed by point number "2", are "a2, b2".

[0037] Figure 9 shows that the coordinates of point P1, managed by point number "1", in the 3D reconstruction model Mr are "x1, y1, z1", and the coordinates of point P2, managed by point number "2", in the 3D reconstruction model Mr are "x2, y2, z2".

[0038] In the example shown in Figure 7, one video frame is associated with each point managed by a point number, but the present invention is not limited to this. Multiple video frames may be associated with a single point.

[0039] Thus, the 3D reconstruction processing unit 1 not only generates the 3D reconstruction model Mr, but also generates a video point cloud correspondence table T that associates each point in the video data D with the 3D reconstruction model Mr.

[0040] <Overview of video search processing> Next, with reference to Figure 10, an overview of the video search process performed by the video search unit 4 will be described. Figure 10 is a diagram illustrating the overview of the video search process performed by the video search unit 4. Figure 10A shows an example of the point cloud Pc extracted by the masking area extraction unit 3. The point cloud Pc includes point P112, which is assigned point number "112", and point P2243, which is assigned point number "2243". Each other point that makes up the point cloud Pc is also assigned a point cloud number and managed accordingly.

[0041] Figure 10B shows the masking target video point cloud correspondence table Ta, which shows the correspondence between points and the 3D reconstruction model Mr. As shown in Figure 10B, the masking target video point cloud correspondence table Ta has the items "point number", "video time", and "position in video". The video search unit 4 generates the masking target video point cloud correspondence table Ta by extracting the items "point number", "video time", and "position in video" from the video point cloud correspondence table T shown in Figure 7.

[0042] For example, the top record in the masking target video point cloud table Ta indicates that for the point managed by point number "112", the video time is "00:00:24" and the in-video position is "a112,b112". The second record from the top in the masking target video point cloud table Ta indicates that for the point managed by point number "2243", the video time is "00:00:25" and the in-video position is "a2243,b2243".

[0043] Figures 10C and 10D show examples of frames identified by video timestamps. Figure 10C shows that the position of point P112, managed by point number "112", within frame F1 is indicated by the coordinates "a112, b112". Figure 10D shows that the position of point P2243, managed by point number "2243", within frame F2 is indicated by the coordinates "a2243, b2243". Pixels in video data D exist at the specified point locations. Therefore, by the video search unit 4 identifying points within frames identified by video timestamps, pixels in video data D are identified.

[0044] The video search unit 4 generates a masking target video point cloud correspondence table Ta that shows the correspondence between all points included in the point cloud Pc and the 3D reconstruction model Mr. The video search unit 4 then transmits the information on the correspondence between the points and the 3D reconstruction model Mr shown in the masking target video point cloud correspondence table Ta to the masking processing unit 5.

[0045] <Overview of Masking Process> Next, an overview of the masking process performed by the masking processing unit 5 will be described with reference to Figures 11 to 13. Figure 11 is a diagram showing an example of the configuration of a frame identified by the video time. Figure 11 shows an example of frame F1 with video time "00:00:24". This frame is the frame in which a point managed by point number "112" exists in the masking target video point cloud correspondence table Ta (see Figure 9).

[0046] Figure 12 shows an example of a point cloud Pc consisting of points defined in the masking target video point cloud correspondence table Ta. Figure 12 shows frame F1 at video time "00:00:24", and in frame F1, the area where the point cloud Pc is located is indicated by a colored cloud mark. The point cloud Pc is the point cloud of the 3D reconstruction model Mr that overlaps with the masking target area Am set in the CAD model Mc, and the point cloud Pc includes the point managed by point number "112" in the masking target video point cloud correspondence table Ta.

[0047] The masking processing unit 5 masks the image of the point cloud Pc at the position corresponding to the masking target region Am by converting, for example, the pixels of each point in the point cloud Pc managed in the point cloud corresponding table Ta to be masked to black.

[0048] Figure 13 shows an example of a frame that has been masked by the masking processing unit 5. Figure 13 shows an example in which the image of the point cloud Pc has been masked by black circles by the masking processing unit 5. The masking processing unit 5 performs masking on all frames that make up the video data D and outputs the video data Dm after masking.

[0049] Furthermore, the masking processing unit 5 may, as a masking process, perform a process to convert the corresponding pixels to a random color or to white, instead of converting them to black. In addition, the masking processing unit 5 may perform a mosaic process, such as dividing the pixel group of the point cloud Pc corresponding to the masking target area Am into certain regions and converting the pixels in each divided region using the average value of the color of the pixel group contained in each region. Moreover, the masking processing unit 5 may transform the image of the point cloud Pc corresponding to the masking area Am so that pseudo-image information different from the image shown in the original video data D is formed.

[0050] Furthermore, the masking processing unit 5 may target not only the image of the point cloud Pc corresponding to the masking region Am, but also a predetermined number of surrounding images for masking. By performing such processing, it is possible to prevent the masked object from being inferred from the shape of the masked area.

[0051] <Video processing support using video processing support device> Next, with reference to Figure 14, a method for supporting video processing by the video processing support device 100 will be described. Figure 14 is a flowchart showing an example of the procedure for video processing support by the video processing support device 100.

[0052] First, the 3D reconstruction processing unit 1 acquires video data D of a space from a storage unit (non-volatile storage 52 in Figure 15) etc. (Step S1). Next, the 3D reconstruction processing unit 1 generates a 3D reconstruction model Mr composed of 3D point cloud information from the acquired video data D (Step S2). Next, the 3D reconstruction processing unit 1 generates a video point cloud correspondence table T (see Figure 7) that associates each point constituting the 3D reconstruction model Mr with the frames and pixels in the video data D (Step S3).

[0053] Next, the alignment processing unit 2 aligns the 3D reconstruction model Mr with the CAD model Mc with the masking target area Am already set (step S4). Then, the masking area extraction unit 3 extracts the point cloud of the area in the CAD model Mc that overlaps with the masking target area Am from the 3D reconstruction model Mr and outputs it to the video search unit 4 (step S5).

[0054] Next, the video search unit 4 extracts and outputs information corresponding to the point cloud from the video data D based on the point cloud information extracted by the masking portion extraction unit 3 and the information described in the video point cloud correspondence table T (step S6). Next, the video search unit 4 generates a masking target video point cloud correspondence table Ta that associates each point constituting the extracted point cloud with the frame and position (coordinate) within the frame in the video data D, and outputs it to the masking processing unit 5 (step S7).

[0055] Next, the masking processing unit 5 takes the video point cloud correspondence table Ta and the video data D as input, identifies the relevant locations in the video data D, and performs masking on the identified locations. Then, the masking processing unit 5 outputs the video data Dm after the masking process (step S8). After the processing in step S8, the video processing support processing by the video processing support device 100 is completed.

[0056] <Example of computer hardware configuration> Finally, with reference to Figure 15, the hardware configuration of the video processing support device 100 according to this embodiment will be described. Figure 15 is a block diagram showing an example of the hardware configuration of the video processing support device 100. The computer 50 shown in Figure 15 is hardware used as a so-called computer.

[0057] As shown in Figure 15, the computer 50 comprises a control unit 51 connected to bus B, a non-volatile storage 52, a display unit 53, an operation input unit 54, and a communication interface 55.

[0058] The control unit 51 includes a CPU (Central Processing Unit) 511, a ROM (Read Only Memory) 512, and a RAM (Random Access Memory) 513.

[0059] The CPU 511 reads the program code of the software that implements each function according to this embodiment from the ROM 512, loads it into the RAM 513, and executes it. Variables, parameters, etc. that occur during the calculation process are temporarily written to the RAM 513.

[0060] The control unit 51 may be equipped with a processing unit such as an MPU (Micro-Processing Unit) instead of a CPU 511. Alternatively, the control unit 51 may use both a CPU and an MPU.

[0061] Examples of non-volatile storage 52 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), flexible disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, and non-volatile memory cards. This non-volatile storage 52 stores the OS, various parameters, and programs necessary for the computer 50 to function. The programs may also be stored in the ROM 512.

[0062] The display unit 53 is a monitor, for example, an LCD (Liquid Crystal Display), which displays the results of processing performed by the computer 50. The operation input unit 54 is composed of, for example, a keyboard, mouse, touch sensor, etc., and generates operation signals in response to user operations and supplies them to the CPU 511. The display unit 53 and the operation input unit 54 may be integrated as a single touch panel.

[0063] The program is stored in the form of computer-readable program code, and the CPU 511 sequentially executes operations according to the program code. In other words, the ROM 512 or non-volatile storage 52 is used as an example of a computer-readable, non-transient recording medium that stores a program executed by the computer.

[0064] Communication I / F55 can utilize, for example, a NIC (Network Interface Card), enabling the transmission and reception of various types of data with external devices via a network or communication line.

[0065] In the embodiment described above, the area to be masked in the video data D is identified based on a CAD model Mc in which the area to be concealed is predefined, and a 3D reconstruction model Mr generated from the video data D in which the space is captured. Then, masking is performed on the pixels in the area to be masked. Therefore, according to this embodiment, processing of the concealed object in the video can be performed without requiring prior training such as machine learning.

[0066] Furthermore, in this embodiment, since machine learning is not required, tolerance for detection failures, which is necessary when performing machine learning, is also unnecessary. Therefore, double-checking and other measures that may be taken to address detection failures that may occur based on tolerance are also unnecessary, thus eliminating the need for human verification.

[0067] Furthermore, in this embodiment, a CAD model Mc in which the area to be concealed is pre-defined is aligned with a 3D reconstruction model Mr generated from video data D in which the space is captured, and the area to be masked is identified in the video data D after alignment. Therefore, according to this embodiment, it is possible to prevent the occurrence of situations in which things that do not actually need to be concealed are mistakenly detected as items that should be concealed.

[0068] Furthermore, in this embodiment, the masking processing unit 5 automatically performs masking on the video data D. Therefore, according to this embodiment, the time required from the time the space is captured and the video data D is generated until the masked video is made public to a third party can be shortened.

[0069] The embodiments described above are intended to explain the present invention in an easy-to-understand manner, providing a detailed and specific description of each component of the device (video image processing support device), and are not necessarily limited to devices that include all of the described components.

[0070] Furthermore, the control lines or information lines indicated by solid arrows in Figures 1 and 2 are those deemed necessary for explanation and do not necessarily represent all control lines or information lines in the actual product. In practice, it can be assumed that almost all components are interconnected. [Explanation of Symbols]

[0071] 1...3D reconstruction processing unit, 2...Alignment processing unit, 3...Masking area extraction unit, 4...Video search unit, 5...Masking processing unit, 6...Video point cloud compatible table storage unit, 100...Video image processing support device, Mc...CAD model, Mr...3D reconstruction model, T...Video point cloud compatible table

Claims

1. A 3D reconstruction processing unit generates a 3D reconstruction model composed of 3D point cloud information from video data of a captured space, A 3D CAD model representing the space in which the confidential object is set, and a confidential object region extraction unit that extracts the confidential object region in the 3D reconstruction model based on the 3D reconstruction model, The system includes a masking processing unit that applies masking to the region in the video data corresponding to the region identified by the region to be kept confidential. A device for assisting in the processing of moving images.

2. The three-dimensional reconstruction processing unit generates a video point cloud correspondence table that associates each point constituting the three-dimensional reconstruction model with the position information of the point in the video data, and outputs it to the confidential target area extraction unit. The motion image processing support apparatus according to claim 1.

3. In the aforementioned video point cloud correspondence table, each point constituting the three-dimensional reconstruction model is associated with the frame in the video data in which the point exists, the position of the point in the frame, and the position of the point in the three-dimensional reconstruction model. The motion image processing support apparatus according to claim 2.

4. The system further includes a alignment processing unit that aligns the three-dimensional reconstruction model with the three-dimensional CAD model and outputs the aligned three-dimensional reconstruction model and the three-dimensional CAD model to the confidential target area extraction unit. The motion image processing support apparatus according to claim 2.

5. The alignment processing unit aligns the three-dimensional reconstruction model and the three-dimensional CAD model by placing them in the same coordinate space. The motion image processing support apparatus according to claim 4.

6. The confidential target area extraction unit extracts a point cloud present in the confidential target area from the 3D reconstruction model, which has been aligned with the 3D CAD model, and outputs identification information for each point constituting the extracted point cloud. The motion image processing support apparatus according to claim 4.

7. The system further includes a video search unit that extracts information from the video point cloud correspondence table, corresponding to the identification information of each point input from the concealed area extraction unit, regarding the frame in which the point exists in the video data and the position of the point in the frame, and outputs this information to the masking processing unit. The motion image processing support apparatus according to claim 6.

8. The procedure involves a 3D reconstruction processing unit generating a 3D reconstruction model composed of 3D point cloud information from video data of a captured space, and A procedure for a confidential target area extraction unit to extract the confidential target area in the three-dimensional reconstruction model based on a three-dimensional CAD model representing the space in which the confidential target is set, and the three-dimensional reconstruction model. The procedure includes: a masking processing unit performing a masking process on the region in the video data corresponding to the region identified by the region to be kept secret; A method for supporting the processing of moving images.

Images