Data processing apparatus and program

Abstract

To provide a data processing apparatus and a program which allows a computer to function as a data processing apparatus.SOLUTION: A data processing apparatus 100, for transforming moving-image data captured with a camera into still-image data, comprises a moving-image data acquiring section that acquires moving-image data in which an object is captured and a data generating section that transforms information on time-series movement of the object in the moving-image data into an element of a still image and generates still-image data including the element of the still image transformed. The data generating section may generate the still-image data representing the information on time-series movement of the object in the moving-image data by at least any of a pixel value, a feature vector and an edge.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a data processing device and a program. [Background technology]

[0002] Patent Document 1 describes a method for automatically generating training data that reproduces the various swimming movements of fish using musical scores that correspond to existing music pieces. [Prior art document] [Patent documents] [Patent Document 1] Patent No. 7152535 Summary of the Invention [Means for solving the problem]

[0003] According to an embodiment of the present invention, there is provided a data processing device. The data processing device may include a video data acquisition unit that acquires video data of an object. The data processing device may include a data generation unit that converts information on time-series movement of the object in the video data into still image elements and generates still image data including the converted still image elements.

[0004] In the data processing device, the data generation section may generate the still image data that represents information on time-series movement of the object in the video data by at least one of pixel values, feature vectors, and edges.

[0005] In any of the data processing devices, the data generation unit may convert information of changes in the time series positions of multiple objects in the video data into the elements of the still image, and generate the still image data including the elements of the converted still image.

[0006] In any of the data processing devices, the data generation unit may convert information of time-series shape changes of the object in the video data into the elements of the still image, and generate the still image data including the elements of the converted still image.

[0007] In any of the data processing devices, the data generation section may convert information on time-series movement of the object in the video data into text elements, and generate text data including the converted text elements.

[0008] In any of the data processing devices, the data generation unit may convert information on the time-series movement of the object in the video data into genome elements, and generate genome data including the converted genome elements.

[0009] Any of the data processing devices may further include a classification result acquisition unit that inputs the still image data generated by the data generation unit into a still image classification network trained to classify still image data and acquires the classification result output from the still image classification network, and an alert output unit that outputs an alert when the classification result acquired by the classification result acquisition unit satisfies a predetermined condition.

[0010] Any of the data processing devices may further include a conversion data acquisition unit that acquires still image data including still image elements converted from information about the time-series movement of an object in the video data, and a video data generation unit that generates the video data from the still image data.

[0011] According to an embodiment of the present invention, there is provided a data processing device. The data processing device may include a still image data acquisition unit that acquires still image data including still image elements converted from information on time-series movement of an object in video data. The data processing device may include a video data generation unit that generates the video data from the still image data.

[0012] According to an embodiment of the present invention, there is provided a data processing device. The data processing device may include a video data acquisition unit that acquires video data of an object. The data processing device may include a data generation unit that converts information on time-series movement of the object in the video data into text elements and generates text data including the converted text elements.

[0013] According to an embodiment of the present invention, there is provided a data processing device. The data processing device may include a video data acquisition unit that acquires video data of an object. The data processing device may include a data generation unit that converts information on time-series movement of the object in the video data into genome elements and generates genome data including the converted genome elements.

[0014] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the data processing device.

[0015] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0016] [Figure 1] An example of a data processing device 100 is shown diagrammatically. [Diagram 2] 1 is an explanatory diagram for explaining a conversion process for converting moving image data 200 into still image data 300. FIG. [Diagram 3] 1 is an explanatory diagram for explaining a conversion process for converting a plurality of pieces of moving image data 200 into still image data 300. FIG. [Figure 4] 1 is an explanatory diagram for explaining a conversion process for converting a plurality of pieces of moving image data 200 into still image data 300. FIG. [Diagram 5] 1 is an explanatory diagram for explaining a conversion process for converting video data 200 into text data 400. FIG. [Figure 6] 1 is an explanatory diagram for explaining a conversion process for converting video data 200 into text data 400. FIG. [Figure 7] 1 is an explanatory diagram for explaining a conversion process for converting video data 200 into genome data 500. FIG. [Figure 8] 1 illustrates an example of a functional configuration of a data processing device 100. [Figure 9] An example of a hardware configuration of a computer 1200 functioning as the data processing device 100 is shown in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0018] In fish farming, understanding the condition of fish is very important to prevent waste of feed, mass death, disease, etc. However, it is difficult to monitor a fish farm around the clock, and a monitoring system that automatically analyzes videos of the fish farm and provides analysis results is desired. Such a monitoring system is needed not only for fish, but also for any object. Such a monitoring system needs to analyze video data including four-dimensional information having three-dimensional space and time, but analyzing the video data as it is imposes a very high processing load. In contrast, the data processing device 100 according to this embodiment converts the video data into data of a lower dimension. For example, the data processing device 100 converts the video data into two-dimensional data.

[0019] As a specific example, the data processing device 100 converts video data into still image data. The data processing device 100 may convert information included in the video data into still image elements to convert the video data into still image data. For example, the data processing device 100 performs object detection on the video data and converts information on the time-series movement of the object into still image elements to convert the video data into still image data. The data processing device 100 may convert the video data into still image data according to conversion information including a conversion rule. The data processing device 100 may realize monitoring of the object by analyzing the converted still image data. This can reduce the processing load. In addition, neural networks that analyze still image data have been studied very widely, and many neural networks already exist, and it can be made possible to utilize such neural networks.

[0020] As a specific example, the data processing device 100 converts video data into text data. The data processing device 100 may convert information contained in the video data into text elements, thereby converting the video data into text data. For example, the data processing device 100 performs object detection on the video data, and converts information on the time-series movement of the object into text elements, thereby converting the video data into text data. The data processing device 100 may convert the video data into text data according to conversion information including a conversion rule. The data processing device 100 may realize object monitoring by analyzing the converted text data. This can reduce the processing load. In addition, neural networks that analyze text data have been studied very widely, and many neural networks already exist, and it can be made possible to utilize such neural networks.

[0021] The data processing device 100 may convert the video data into data in which specific characters are listed or data in which numbers are listed. The data processing device 100 converts the video data into genome data, for example. The genome data may be data composed of a base sequence. The data processing device 100 may convert the video data into genome data by converting information contained in the video data into genome elements. For example, the data processing device 100 performs object detection on the video data and converts information on the time-series movement of the object into genome elements, thereby converting the video data into genome data. The data processing device 100 may convert the video data into genome data according to conversion information including a conversion rule. The data processing device 100 may realize monitoring of the object by analyzing the converted genome data. This can reduce the processing load. In addition, neural networks that analyze genome data have been studied very widely, and many neural networks already exist, and it is possible to reuse such neural networks.

[0022] 1 is a schematic diagram of an example of a data processing device 100. The data processing device 100 may acquire video data of an object, and convert the acquired video data into data having a lower dimension than the video data (the converted data may be referred to as converted data). Examples of data having a lower dimension than video data include still image data, text data, and genome data.

[0023] The object may be anything that changes over time. The object may be one whose position changes over time, one whose shape changes over time, or one whose position and shape change over time. The object may be one whose shape does not change over time but only whose position changes. The object may be one whose shape changes over time but not whose position changes over time.

[0024] The object may be a target of monitoring. In the present embodiment, a case will be described in which the object is a fish and the target of monitoring is a school of fish, but the present invention is not limited to this.

[0025] The data processing device 100 may acquire video data captured by the camera 102. The camera 102 may be built into the data processing device 100. The camera 102 may be externally attached to the data processing device 100. The camera 102 may be connected to the data processing device 100 via any network.

[0026] The data processing device 100 may receive video data from the communication terminal 30. The communication terminal 30 transmits, for example, video data captured by a camera 32 to the data processing device 100. The camera 32 may be built into the communication terminal 30. The camera 32 may be externally attached to the communication terminal 30. The camera 32 may be connected to the communication terminal 30 via any network.

[0027] The communication terminal 30 may be a PC (Personal Computer), a tablet terminal, a smartphone, or the like. The data processing device 100 and the communication terminal 30 may communicate with each other via a network 20. The network 20 may include the Internet. The network 20 may include a LAN (Local Area Network). The network 20 may include a mobile communication network. The mobile communication network may conform to any of the following communication methods: 5G (5th Generation) communication method, LTE (Long Term Evolution) communication method, 3G (3rd Generation) communication method, and 6G (6th Generation) communication method or later.

[0028] The data processing device 100 may pre-store a classification network trained to classify the converted data. The data processing device 100 may input the converted data to the classification network and obtain a classification result output from the classification network. The data processing device 100 may output an alert when the classification result satisfies a predetermined condition.

[0029] As a specific example, the data processing device 100 acquires video data of a school of fish in a fish farm, converts information on the time series movements of multiple fish in the video data into still image elements, and generates still image data including the converted still image elements. The data processing device 100 converts, for example, information on the time series changes in the positions of multiple fish in the video data into still image elements, and generates still image data including the converted still image elements. The data processing device 100 may generate still image data for any time unit, such as seconds, minutes, hours, days, weeks, months, years, etc.

[0030] For example, the data processing device 100 generates still image data in one-hour units. In this case, one still image data includes the characteristics of the change in the time-series positions of the multiple fishes during one hour. For example, five still image data are generated from five hours of video data. If the multiple fishes are not particularly abnormal and are swimming normally during the five hours, the five still image data will be images similar to each other. In contrast, if the multiple fishes swim normally for four hours but make a violent movement different from the previous movements in the last hour, the four still image data corresponding to the first four hours will be images similar to each other, but the one still image data corresponding to the last hour may be an image showing characteristics different from the four still image data. In other words, by classifying the still image data, it is possible to monitor the change in the state of the school of fish.

[0031] The data processing device 100 may generate a still image classification network that classifies still image data by executing machine learning using a large amount of generated still image data. The data processing device 100 may input newly generated still image data to the generated still image classification network, thereby classifying the still image data.

[0032] The data processing device 100 may use an existing still image classification network. For example, the data processing device 100 acquires and stores in advance a still image classification network generated by another device. Then, the data processing device 100 inputs still image data generated thereafter into the still image classification network, thereby classifying the still image data.

[0033] The data processing device 100 may further include a function of generating moving image data from the converted data. The data processing device 100 may generate moving image data from the converted data generated by itself. The data processing device 100 may convert the converted data into moving image data by using conversion information used when converting the moving image data into the converted data.

[0034] The data processing device 100 may generate video data from converted data generated by another device. The data processing device 100 may receive conversion information used when converting video data into converted data from the other device that generated the converted data, and may convert the converted data received from the other device into video data by using the conversion information.

[0035] The data processing device 100 may have a function of generating moving image data from converted data, without having a function of generating converted data from moving image data.

[0036] 2 is an explanatory diagram for explaining the conversion process of converting video data 200 into still image data 300. Here, a case will be explained in which the camera 102 converts video data 200 capturing an image of a school of fish in the fish farm 40 into still image data 300.

[0037] The data processing device 100 acquires video data 200 from the camera 102. The data processing device 100 converts information on the time-series movement of the school of fish in the video data 200 into still image elements, and generates still image data 300 including the converted still image elements.

[0038] For example, the data processing device 100 generates one portion 302 constituting the still image data 300 from one frame of the moving image data 200. The data processing device 100 may generate still image data 300 consisting of the multiple portions 302 by generating multiple portions 302 each corresponding to a respective one of the multiple frames of the moving image data 200. Note that the data processing device 100 may generate one portion 302 from multiple frames of the moving image data 200.

[0039] The data processing device 100 may generate the still image data 300 for each arbitrary period. For example, when generating the still image data 300 for each day, the data processing device 100 generates the still image data 300 corresponding to one day by generating a plurality of portions 302 from frames of the video data 200 for one day.

[0040] The data processing device 100 may generate still image data 300 from a plurality of moving image data 200 captured in parallel by a plurality of cameras.

[0041] 3 and 4 are explanatory diagrams for explaining the conversion process of converting a plurality of video data 200 into still image data 300. Here, a case will be described in which six video data 200 capturing images of a school of fish in a fish farm 40 from six directions are converted into still image data 300. In the example shown in Figs. 3 and 4, the school of fish in the fish farm 40 is captured from four directions, namely above, below, and side (side A, side B, side C, side D).

[0042] FIG. 3 illustrates, at a certain timing, frame 211 of video data 210 captured from above, frame 221 of video data 220 captured from below, frame 231 of video data 230 captured from side A, frame 241 of video data 240 captured from side B, frame 251 of video data 250 captured from side C, and frame 261 of video data 260 captured from side D.

[0043] The data processing device 100, for example, generates a portion 311 from frame 211, generates a portion 321 from frame 221, generates a portion 331 from frame 231, generates a portion 341 from frame 241, generates a portion 351 from frame 251, and generates a portion 361 from frame 261. The data processing device 100 connects these as shown in Fig. 4. The data processing device 100 similarly generates and connects still image portions for the next frame. By repeating this, the data processing device 100 generates still image data 300.

[0044] The data processing device 100 may generate one portion from a plurality of consecutive frames, instead of generating one portion from one frame as described above. For example, the data processing device 100 generates one portion 311 from 10 consecutive frames of the video data 210 captured from above, generates one portion 321 from 10 consecutive frames of the video data 220 captured from below, generates one portion 331 from 10 consecutive frames of the video data 230 captured from side A, generates one portion 341 from 10 consecutive frames of the video data 240 captured from side B, generates one portion 351 from 10 consecutive frames of the video data 250 captured from side C, and generates one portion 361 from 10 consecutive frames of the video data 260 captured from side D. 10 frames is just an example, and the number of frames may be other numbers.

[0045] 5 is an explanatory diagram for explaining the conversion process of converting the video data 200 into text data 400. Here, a case will be described in which the video data 200, which is an image of a school of fish in the fish farm 40 captured by the camera 102, is converted into text data 400.

[0046] The data processing device 100 acquires video data 200 from the camera 102. The data processing device 100 converts information on the time-series movement of the school of fish in the video data 200 into text elements, and generates text data 400 including the converted text elements.

[0047] For example, the data processing device 100 generates one portion 402 constituting the text data 400 from one frame of the video data 200. The data processing device 100 may generate the text data 400 consisting of the multiple portions 402 by generating multiple portions 402 each corresponding to a multiple frame of the video data 200. Note that the data processing device 100 may generate one portion 402 from multiple frames of the video data 200.

[0048] The data processing device 100 may generate the text data 400 for each arbitrary period. For example, when generating the text data 400 for each day, the data processing device 100 generates the text data 400 corresponding to one day by generating a plurality of parts 402 from frames of the video data 200 for one day.

[0049] The data processing device 100 may generate text data 400 from multiple video data 200 captured in parallel by multiple cameras. Here, a case will be described in which six video data 200 capturing images of a school of fish in a fish farm 40 from six directions as shown in Fig. 3 are converted into text data 400.

[0050] The data processing device 100, for example, generates a portion 411 from frame 211, generates a portion 421 from frame 221, generates a portion 431 from frame 231, generates a portion 441 from frame 241, generates a portion 451 from frame 251, and generates a portion 461 from frame 261. The data processing device 100 concatenates these as shown in Fig. 6. The data processing device 100 similarly generates and concatenates portions of text for the next frame. By repeating this, the data processing device 100 generates text data 400.

[0051] The data processing device 100 may generate one portion from a plurality of consecutive frames, instead of generating one portion from one frame as described above. For example, the data processing device 100 generates one portion 411 from 10 consecutive frames of the video data 210 captured from above, generates one portion 421 from 10 consecutive frames of the video data 220 captured from below, generates one portion 431 from 10 consecutive frames of the video data 230 captured from side A, generates one portion 441 from 10 consecutive frames of the video data 240 captured from side B, generates one portion 451 from 10 consecutive frames of the video data 250 captured from side C, and generates one portion 461 from 10 consecutive frames of the video data 260 captured from side D. 10 frames is just an example, and the number of frames may be other numbers.

[0052] 7 is an explanatory diagram for explaining the conversion process of converting the video data 200 into genome data 500. Here, a case will be described in which the video data 200, which is an image of a school of fish in the fish farm 40 captured by the camera 102, is converted into genome data 500.

[0053] The data processing device 100 acquires video data 200 from the camera 102. The data processing device 100 converts information on the time-series movement of the school of fish in the video data 200 into genome elements, and generates genome data 500 including the converted genome elements.

[0054] For example, the data processing device 100 generates one portion constituting the genome data 500 from one frame of the video data 200. The data processing device 100 may generate the genome data 500 consisting of multiple portions by generating multiple portions each corresponding to each of the multiple frames of the video data 200. Note that the data processing device 100 may generate one portion 402 from multiple frames of the video data 200.

[0055] The data processing device 100 may generate the genome data 500 for each arbitrary period. For example, when generating the genome data 500 for each day, the data processing device 100 generates the genome data 500 corresponding to one day by generating a plurality of parts from frames of the video data 200 for one day.

[0056] The data processing device 100 may generate genome data 500 from a plurality of video data 200 captured in parallel by a plurality of cameras, similar to the still image data 300 and text data 400.

[0057] 8 illustrates an example of a functional configuration of the data processing device 100. The data processing device 100 includes a conversion information storage unit 112, a model storage unit 114, a video data acquisition unit 116, a data generation unit 118, a classification result acquisition unit 120, a model generation unit 122, an output unit 130, a conversion data acquisition unit 142, and a video data generation unit 144.

[0058] The conversion information storage unit 112 stores conversion information including a conversion rule for converting the video data 200. The conversion information storage unit 112 may store conversion information registered in advance. For example, the conversion information storage unit 112 stores conversion information including a conversion rule for converting the video data 200 into still image data 300. For example, the conversion information storage unit 112 stores conversion information including a conversion rule for converting the video data 200 into text data 400. For example, the conversion information storage unit 112 stores conversion information including a conversion rule for converting the video data 200 into genome data 500.

[0059] The model storage unit 114 stores a classification network trained to classify the converted data. The model storage unit 114 may store a classification network registered in advance. The model storage unit 114 stores, for example, a classification network generated by another device. For example, the conversion information storage unit 112 stores a still image classification network. For example, the conversion information storage unit 112 stores a text classification network. For example, the conversion information storage unit 112 stores a genome classification network.

[0060] The video data acquisition unit 116 acquires video data 200 that captures an object. The video data acquisition unit 116 acquires the video data 200 to be converted. The video data acquisition unit 116 may receive the video data 200 from the camera 102. The video data acquisition unit 116 may receive the video data 200 from the communication terminal 30. The video data acquisition unit 116 may receive the video data 200 from another device. The video data acquisition unit 116 may acquire the video data 200 from a portable storage device or the like.

[0061] The video data 200 may be accompanied by a detection result of an object. The video data 200 may be accompanied by information on the three-dimensional position of the object. The information on the three-dimensional position of the object may be specified by a method using an output from a distance measuring sensor arranged separately from the camera 102 or the camera 32, a method using an image analysis result of the camera 102 or the camera 32, or the like. Note that the method is not limited to these, and any method may be used as long as it can specify the three-dimensional position of the object.

[0062] The data generation unit 118 converts the video data 200 acquired by the video data acquisition unit 116 to generate converted data. The data generation unit 118 may generate the converted data from the video data 200 by using the conversion information stored in the conversion information storage unit 112.

[0063] For example, the data generating unit 118 generates still image data 300 from the video data 200. The data generating unit 118 may convert information on the time-series movement of an object in the video data 200 into still image elements, and generate still image data 300 including the converted still image elements. The data generating unit 118 may generate still image data 300 that represents information on the time-series movement of an object in the video data 200 by at least one of pixel values, feature vectors, and edges. Thus, examples of still image elements include, but are not limited to, pixel values, feature vectors, and edges.

[0064] The data generator 118 may convert information on time-series changes in the positions of a plurality of objects in the video data 200 into still image elements, and generate still image data 300 including the converted still image elements.

[0065] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series positional changes of a plurality of objects in the video data 200 by pixel values. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into pixel value information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects included in the frame as pixel values ​​of three consecutive pixels. As a result, for a portion of the still image data 300 corresponding to the frame, the pixel values ​​of the three consecutive pixels indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate still image data 300 corresponding to the video data 200 by similarly assigning the pixel values ​​for other frames of the video data 200. The method of converting the information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects into pixel values ​​is not limited to this. The information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects may be compressed and converted into pixel values ​​using an existing compression technology. The frame division may be expressed by the arrangement, or one or more pixel values ​​indicating the frame division may be used. Methods other than these may also be used for the frame division.

[0066] Grayscale may be used as pixel values. This results in the generation of grayscale still image data 300 that represents information on changes in the positions of multiple objects in the video data 200 over time. When multiple objects in the video data 200 are moving in a similar manner, the still image data 300 may have an appearance in which the same pattern is repeated. Furthermore, when multiple objects in the video data 200 are moving violently for a certain period of time, the still image data 300 may have an appearance in which the part becomes darker, lighter, or changes more violently, thereby indicating that the part is different from the other parts.

[0067] Color may be used as the pixel value. This allows information on the change in the positions of multiple objects in time series to be expressed more efficiently than when grayscale is used. When color is used, color still image data 300 is generated, which represents information on the change in the positions of multiple objects in time series in the video data 200. When multiple objects in the video data 200 are moving in a similar manner, the still image data 300 may have the same color tone. Furthermore, when multiple objects in the video data 200 are moving intensely for a certain period of time, the still image data 300 may have an appearance that indicates that the part is different from the other parts by making the color of the part darker or lighter, or by making the color tone change more drastically.

[0068] The data generating unit 118 generates still image data 300 that expresses, for example, information on the change in the positions of multiple objects in the video data 200 over time by a feature vector. The data generating unit 118 converts, for example, information on the three-dimensional position of an object for each frame into information on a feature vector. The data generating unit 118 may use vectors that quantify position coordinates, velocity change amount, and direction, or may use a method of learning the features themselves, as a method of expressing a feature vector. When there are a large number of target objects, such as a school of fish, the feature vector may be generated after taking statistics on the positions of multiple objects over time. In addition, statistics may be taken from the feature vectors, and these may be used as feature vectors in a higher layer.

[0069] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series positional changes of multiple objects in the video data 200 using edges. The data generating unit 118 converts, for example, information on the three-dimensional position of an object for each frame into edge information. The data generating unit 118 may realize conversion between information on time-series positional changes of multiple objects and edge information using, for example, a method of binarizing and differentiating, the Canny method, a method of learning edges or silhouettes using a DNN (Deep Neural Network), or the like.

[0070] The data generator 118 may convert information on time-series shape changes of an object in the video data 200 into still image elements, and generate still image data 300 including the converted still image elements.

[0071] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in the video data 200 using pixel values. The data generating unit 118 converts, for example, information on three-dimensional positions of a plurality of structural points of an object for each frame into information on pixel values. For example, if the object is a living thing or a robot, the structural points of the object may be the positions of joints. If the object does not have joints, the structural points of the object may be each point of the object that affects the movement of the object.

[0072] As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X-coordinate, Y-coordinate, and Z-coordinate of each of a plurality of structural points of an object included in the frame as pixel values ​​of three consecutive pixels. As a result, for a portion of the still image data 300 corresponding to the frame, the pixel values ​​of three consecutive pixels indicate the X-coordinate, Y-coordinate, and Z-coordinate of the object. The data generating unit 118 may generate still image data 300 corresponding to the video data 200 by similarly assigning pixel values ​​for other frames of the video data 200. The method of converting the information of the X-coordinate, Y-coordinate, and Z-coordinate of each of the plurality of objects into pixel values ​​is not limited to this. The information of the X-coordinate, Y-coordinate, and Z-coordinate of each of the plurality of objects may be compressed and converted into pixel values ​​using an existing compression technique. The frame division may be expressed by arrangement, or one or more pixel values ​​indicating the frame division may be used. Methods other than these may be used for the frame division.

[0073] Grayscale may be used as pixel values. This results in the generation of grayscale still image data 300 that represents information on time-series shape changes of an object in the video data 200. When an object in the video data 200 moves in a similar manner, the still image data 300 may have an appearance in which a similar pattern is repeated. Furthermore, when an object in the video data 200 moves violently for a certain period of time, the still image data 300 may have an appearance in which the part becomes darker, lighter, or changes violently, thereby indicating that the part is different from the other parts.

[0074] Color may be used as the pixel value. This allows information on the time-series shape change of an object to be expressed more efficiently than when grayscale is used. When color is used, color still image data 300 is generated, which represents information on the time-series shape change of an object in the video data 200. When objects in the video data 200 are moving in the same way, the still image data 300 may have the same color tone. Furthermore, when an object in the video data 200 is moving violently for a certain period of time, the still image data 300 may have an appearance that indicates that the object is different from other parts by making the color of the object darker or lighter or by making the color tone change more violently.

[0075] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in the video data 200 using a feature vector. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into information on the feature vector.

[0076] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in the video data 200 using edges. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into edge information.

[0077] For example, the data generating unit 118 generates still image data 300 from a plurality of video data 200 obtained by capturing an object from a plurality of different directions. Here, differences from the case where still image data 300 is generated from one video data 200 will be mainly described. The data generating unit 118 may convert information on the time-series movement of an object in the plurality of video data 200 into still image elements, and generate still image data 300 including the converted still image elements. The data generating unit 118 may generate still image data 300 that represents information on the time-series movement of an object in the plurality of video data 200 by at least one of pixel values, feature vectors, and edges.

[0078] The data generating section 118 may convert information on time-series positional changes of a plurality of objects in a plurality of video data 200 into still image elements, and generate still image data 300 including the converted still image elements.

[0079] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200 by pixel values. For example, the data generating unit 118 converts, for each of the plurality of video data 200, information on the three-dimensional position where an object exists for each frame into pixel value information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects included in the frame as pixel values ​​of three consecutive pixels. As a result, for a portion of the still image data 300 corresponding to the frame, the pixel values ​​of three consecutive pixels indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate still image data 300 corresponding to the video data 200 by similarly assigning pixel values ​​for other frames of the video data 200.

[0080] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200, using feature vectors. The data generating unit 118 converts, for example, for each of the plurality of video data 200, information on the three-dimensional position at which an object exists for each frame into feature vector information.

[0081] The data generating unit 118 generates still image data 300 that expresses, for example, information on changes in the positions of a plurality of objects in a plurality of moving image data 200 by using edges. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame of each of the plurality of moving image data 200 into edge information.

[0082] The data generator 118 may convert information on time-series shape changes of objects in multiple video data 200 into still image elements, and generate still image data 300 including the converted still image elements.

[0083] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 by pixel values. For example, the data generating unit 118 converts, for each of the plurality of video data 200, information on three-dimensional positions where a plurality of structural points of an object exist for each frame into pixel value information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the structural points of an object included in the frame as pixel values ​​of three consecutive pixels. As a result, for a portion of the still image data 300 corresponding to the frame, the pixel values ​​of three consecutive pixels indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate still image data 300 corresponding to the video data 200 by similarly assigning pixel values ​​for other frames of the video data 200.

[0084] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 using a feature vector. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame of the plurality of video data 200 into information on the feature vector.

[0085] The data generating unit 118 generates still image data 300 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 using edges. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame of the plurality of video data 200 into edge information.

[0086] For example, the data generating unit 118 generates text data 400 from the video data 200. The data generating unit 118 may convert information on the time-series movement of an object in the video data 200 into text elements, and generate text data 400 including the converted text elements. The data generating unit 118 may generate text data 400 that represents information on the time-series movement of an object in the video data 200 by at least one of characters, words, fonts, and ASCII codes. In this way, examples of text elements include, but are not limited to, characters, words, fonts, and ASCII codes.

[0087] The data generating section 118 may convert information on changes in the time series positions of a plurality of objects in the video data 200 into text elements, and generate text data 400 including the converted text elements.

[0088] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in the video data 200 by characters. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into character information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects included in the frame as three consecutive characters. As a result, for a portion of the still image data 300 corresponding to the frame, three consecutive characters indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate text data 400 corresponding to the video data 200 by similarly assigning the X coordinate, Y coordinate, and Z coordinate of each of the other frames of the video data 200 as characters. The method of converting the information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects into characters is not limited to this. The information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects may be compressed and converted into characters using an existing compression technology. The frame division may be expressed by the arrangement, or one or more characters may be used to indicate the frame division. Methods other than these may also be used to indicate the frame division.

[0089] Japanese characters may be used as the characters. In this way, Japanese text data 400 is generated, which represents information on changes in the time-series positions of a plurality of objects in the video data 200. When a plurality of objects in the video data 200 are moving in a similar manner, the text data 400 may have similar content, with similar characters being repeated. Furthermore, when a plurality of objects in the video data 200 are moving violently for a certain period of time, the text data 400 may use characters different from those used in other portions. Characters of a country other than Japan may be used as the characters. Furthermore, characters of a plurality of countries may be mixed as characters.

[0090] The data generating unit 118 generates text data 400 that expresses, by words, information on time-series positional changes of a plurality of objects in the video data 200. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into word information.

[0091] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in the video data 200 in ASCII code. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into ASCII code information.

[0092] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in the video data 200 using at least one of characters and words and a font. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into at least one of characters and words and font information.

[0093] The data generating unit 118 may convert information on time-series shape changes of objects in the video data 200 into text elements, and generate text data 400 including the converted text elements.

[0094] The data generating unit 118 generates text data 400 that expresses, by characters, for example, information on time-series shape changes of an object in the video data 200. The data generating unit 118 converts, for example, information on three-dimensional positions where multiple structural points of an object exist for each frame into character information.

[0095] As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X-coordinate, Y-coordinate, and Z-coordinate of each of the multiple structural points of an object included in the frame as three consecutive characters. As a result, for a portion of the text data 400 corresponding to the frame, the three consecutive characters indicate the X-coordinate, Y-coordinate, and Z-coordinate of the object. The data generating unit 118 may generate text data 400 corresponding to the video data 200 by similarly assigning the X-coordinate, Y-coordinate, and Z-coordinate of each of the multiple structural points of the object as characters. The method of converting the information of the X-coordinate, Y-coordinate, and Z-coordinate of each of the multiple structural points of the object into characters is not limited to this. The information of the X-coordinate, Y-coordinate, and Z-coordinate of each of the multiple structural points of the object may be compressed and converted into characters using an existing compression technique. The frame division may be expressed by arrangement, or one or more characters indicating the frame division may be used. Methods other than these may be used for the frame division.

[0096] Japanese characters may be used as the characters. In this way, Japanese text data 400 is generated, which represents information on time-series shape changes of an object in the video data 200. When an object in the video data 200 moves in a similar manner, the text data 400 may have similar content, with similar characters being repeated. Furthermore, when an object in the video data 200 moves violently for a certain period of time, the text data 400 may use characters different from those used in other portions. Characters of a country other than Japan may be used as the characters. Furthermore, characters of a plurality of countries may be mixed as the characters.

[0097] The data generating unit 118 generates text data 400 that expresses, by words, information on time-series shape changes of an object in the video data 200. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into word information.

[0098] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series shape changes of an object in the video data 200 in ASCII code. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into ASCII code information.

[0099] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series shape changes of an object in the video data 200 using at least one of characters and words and a font. The data generating unit 118 converts, for example, information on a three-dimensional position where an object exists for each frame into at least one of characters and words and font information.

[0100] For example, the data generating unit 118 generates text data 400 from a plurality of video data 200 in which an object is captured from a plurality of different directions. Here, differences from the case where text data 400 is generated from one video data 200 will be mainly described. The data generating unit 118 may convert information on the time-series movement of an object in the plurality of video data 200 into text elements, and generate text data 400 including the converted text elements. The data generating unit 118 may generate text data 400 that expresses information on the time-series movement of an object in the plurality of video data 200 by at least one of characters, words, fonts, and ASCII codes.

[0101] The data generating section 118 may convert information on changes in the time series positions of a plurality of objects in a plurality of video data 200 into text elements, and generate text data 400 including the converted text elements.

[0102] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200 by characters. For example, the data generating unit 118 converts, for each of the plurality of video data 200, information on the three-dimensional position where an object exists for each frame into character information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects included in the frame as three consecutive characters. As a result, for a portion of the still image data 300 corresponding to the frame, the three consecutive characters indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate text data 400 corresponding to the video data 200 by similarly assigning the characters to other frames of the video data 200.

[0103] The data generating unit 118 generates text data 400 expressing, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200 using words. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame of each of the plurality of video data 200 into word information.

[0104] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200, in ASCII code. The data generating unit 118 converts, for example, for each of the plurality of video data 200, information on the three-dimensional position at which an object exists for each frame, into ASCII code information.

[0105] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200, by at least one of characters and words and a font. For example, the data generating unit 118 converts information on the three-dimensional position where an object exists for each frame of the plurality of video data 200, into at least one of characters and words and font information.

[0106] The data generating unit 118 may convert information on time-series shape changes of objects in multiple video data 200 into text elements, and generate text data 400 including the converted text elements.

[0107] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 by characters. For example, the data generating unit 118 converts, for each of the plurality of video data 200, information on three-dimensional positions where a plurality of structural points of an object exist for each frame into character information. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the plurality of structural points of an object included in the frame as three consecutive characters. As a result, for a portion of the text data 400 corresponding to the frame, the three consecutive characters indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate text data 400 corresponding to the video data 200 by similarly assigning the X coordinate, Y coordinate, and Z coordinate of the object as characters for other frames of the video data 200.

[0108] The data generating unit 118 generates text data 400 that expresses, by words, information on time-series shape changes of an object in a plurality of video data 200. The data generating unit 118 converts, for example, information on a three-dimensional position where an object exists for each frame of the plurality of video data 200 into information on words.

[0109] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 in ASCII code. The data generating unit 118 converts, for example, information on a three-dimensional position where an object exists for each frame of the plurality of video data 200 into ASCII code information.

[0110] The data generating unit 118 generates text data 400 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 using at least one of characters and words and a font. For example, the data generating unit 118 converts information on a three-dimensional position where an object exists for each frame of the plurality of video data 200 into at least one of characters and words and font information.

[0111] For example, the data generation unit 118 generates genome data 500 from the video data 200. The data generation unit 118 may convert information on the time series movement of an object in the video data 200 into genome elements, and generate genome data 500 including the converted genome elements. The data generation unit 118 may generate genome data 500 that represents information on the time series movement of an object in the video data 200 by a base sequence.

[0112] The data generation unit 118 may convert information on time-series changes in the positions of a plurality of objects in the video data 200 into genome elements, and generate genome data 500 including the converted genome elements.

[0113] The data generating unit 118 generates genome data 500 that expresses, for example, information on the change in the positions of a plurality of objects in the video data 200 over time by a base sequence. The data generating unit 118 converts, for example, information on the three-dimensional position where an object exists for each frame into information on a base sequence. As a specific example, the data generating unit 118 assigns, for one frame of the video data 200, the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects included in the frame as a base sequence. As a result, for a portion of the genome data 500 corresponding to the frame, a continuous base sequence indicates the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate genome data 500 corresponding to the video data 200 by similarly assigning the base sequences for other frames of the video data 200. The method of converting the information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects into a base sequence is not limited to this. The information on the X coordinate, Y coordinate, and Z coordinate of each of the plurality of objects may be compressed and converted into a base sequence using an existing compression technology. The frame division may be expressed by the arrangement, or a base sequence indicating the frame division may be used. The frame division may also be expressed by other methods.

[0114] The data generation unit 118 may convert information on time-series shape changes of an object in the video data 200 into genome elements, and generate genome data 500 including the converted genome elements.

[0115] The data generating unit 118 generates genome data 500 that expresses, for example, information on time-series shape changes of an object in the video data 200 by a base sequence. The data generating unit 118 converts, for example, information on three-dimensional positions where multiple structural points of an object exist for each frame into information on a base sequence. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of multiple structural points of an object included in the frame as a base sequence. As a result, for a portion of the genome data 500 corresponding to the frame, the base sequence indicates the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate genome data 500 corresponding to the video data 200 by similarly assigning the information as a base sequence for other frames of the video data 200. The method of converting the information on the X coordinate, Y coordinate, and Z coordinate of each of multiple structural points of an object into a base sequence is not limited to this. The information on the X coordinate, Y coordinate, and Z coordinate of each of multiple structural points of an object may be compressed and converted into a base sequence using an existing compression technology. The frame division may be expressed by the arrangement, or a base sequence indicating the frame division may be used. The frame division may also be expressed by other methods.

[0116] For example, the data generating unit 118 generates genome data 500 from a plurality of video data 200 in which an object is captured from a plurality of different directions. Here, differences from the case where genome data 500 is generated from one video data 200 will be mainly described. The data generating unit 118 may convert information on the time series movement of an object in the plurality of video data 200 into genome elements, and generate genome data 500 including the converted genome elements. The data generating unit 118 may generate text data 400 that expresses information on the time series movement of an object in the plurality of video data 200 by a base sequence.

[0117] The data generation section 118 may convert information on time-series positional changes of a plurality of objects in a plurality of video data 200 into genome elements, and generate genome data 500 including the converted genome elements.

[0118] The data generating unit 118 generates genome data 500 that expresses, for example, information on time-series positional changes of a plurality of objects in a plurality of video data 200 by a base sequence. For example, the data generating unit 118 converts information on the three-dimensional position of an object for each frame of the plurality of video data 200 into information on a base sequence. As a specific example, the data generating unit 118 assigns, for one frame of the video data 200, the X coordinate, Y coordinate, and Z coordinate of each of a plurality of objects included in the frame as a base sequence. As a result, for a portion of the still image data 300 corresponding to the frame, the base sequence indicates the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate text data 400 corresponding to the video data 200 by similarly assigning the base sequences for other frames of the video data 200. <Genome data: multiple videos: shape change>

[0119] The data generation unit 118 may convert information on time-series shape changes of an object in a plurality of video data 200 into genome elements, and generate genome data 500 including the converted genome elements.

[0120] The data generating unit 118 generates genome data 500 that expresses, for example, information on time-series shape changes of an object in a plurality of video data 200 by a base sequence. For example, the data generating unit 118 converts, for each of the plurality of video data 200, information on three-dimensional positions where a plurality of structural points of the object exist for each frame into information on a base sequence. As a specific example, for one frame of the video data 200, the data generating unit 118 assigns the X coordinate, Y coordinate, and Z coordinate of each of the structural points of the object included in the frame as a base sequence. As a result, for a portion of the genome data 500 corresponding to the frame, three consecutive characters indicate the X coordinate, Y coordinate, and Z coordinate of the object. The data generating unit 118 can generate genome data 500 corresponding to the video data 200 by similarly assigning the base sequences for other frames of the video data 200.

[0121] The classification result acquisition unit 120 inputs the converted data generated by the data generation unit 118 to the classification network stored in the model storage unit 114, and acquires the classification result output from the classification network. For example, the classification result acquisition unit 120 inputs the still image data 300 generated by the classification result acquisition unit 120 to the still image classification network, and acquires the classification result output from the still image classification network. For example, the classification result acquisition unit 120 inputs the text data 400 generated by the classification result acquisition unit 120 to the text classification network, and acquires the classification result output from the text classification network. For example, the classification result acquisition unit 120 inputs the genome data 500 generated by the classification result acquisition unit 120 to the genome classification network, and acquires the classification result output from the genome classification network.

[0122] The model generation unit 122 generates a classification network that classifies the converted data by executing machine learning using the converted data generated by the data generation unit 118. The model generation unit 122 may store the generated classification network in the model storage unit 114. For example, the model generation unit 122 generates a still image classification network that classifies the still image data by executing machine learning using the still image data 300 generated by the data generation unit 118. For example, the model generation unit 122 generates a text classification network that classifies the text data by executing machine learning using the text data 400 generated by the data generation unit 118. For example, the model generation unit 122 generates a genome classification network that classifies the genome data by executing machine learning using the genome data 500 generated by the data generation unit 118.

[0123] The output unit 130 executes various output controls and may include an alert output unit 132, a display output unit 134, and an audio output unit 136.

[0124] The alert output unit 132 outputs an alert according to the classification result by the classification result acquisition unit 120. The alert output unit 132 may cause the display of the data processing device 100 to display and output alert information, cause the speaker of the data processing device 100 to output an alert as sound, cause the display of the communication terminal 30 to display and output alert information, or cause the speaker of the communication terminal 30 to output an alert as sound.

[0125] For example, the alert output unit 132 outputs an alert when the classification result acquired by the classification result acquisition unit 120 satisfies a predetermined condition. For example, in a situation where the video data acquisition unit 116 continuously acquires the video data 200, the data generation unit 118 continuously generates the converted data, and the classification result acquisition unit 120 continuously acquires the classification results, the alert output unit 132 outputs an alert when the degree of difference between the successive classification results exceeds a predetermined threshold. Also, for example, after a plurality of pieces of converted data generated by the data generation unit 118 are accumulated, the classification result acquisition unit 120 acquires classification results collectively for the plurality of converted data, and the alert output unit 132 outputs an alert when the classification results include a classification result indicating an abnormal value.

[0126] The display output unit 134 may display and output the converted data generated by the data generation unit 118. For example, the display output unit 134 displays and outputs the converted data generated by the data generation unit 118 on a display included in the data processing device 100. For example, the display output unit 134 displays and outputs the converted data generated by the data generation unit 118 on a display included in the communication terminal 30.

[0127] For example, the display output unit 134 displays and outputs the still image data 300 generated by the data generation unit 118. The display output unit 134 may continuously display and output the still image data 300 continuously generated by the data generation unit 118. For example, the display output unit 134 displays and outputs the text data 400 generated by the data generation unit 118. The display output unit 134 may continuously display and output the text data 400 continuously generated by the data generation unit 118. For example, the display output unit 134 displays and outputs the genome data 500 generated by the data generation unit 118. The display output unit 134 may continuously display and output the genome data 500 continuously generated by the data generation unit 118. These can make a viewer who has viewed the display aware that some abnormality may have occurred in the object.

[0128] The display output unit 134 may display and output the classification result acquired by the classification result acquisition unit 120. For example, the display output unit 134 displays and outputs the classification result acquired by the classification result acquisition unit 120 on a display included in the data processing device 100. For example, the display output unit 134 displays and outputs the classification result acquired by the classification result acquisition unit 120 on a display included in the communication terminal 30.

[0129] The audio output unit 136 may output the converted data generated by the data generation unit 118 as audio. The display output unit 134 may output, for example, the text data 400 generated by the data generation unit 118 as audio. The audio output unit 136 may output the text data 400 generated by the data generation unit 118 as audio to a speaker included in the data processing device 100. The audio output unit 136 may output the text data 400 generated by the data generation unit 118 as audio to a speaker included in the communication terminal 30. This makes it possible for a person who hears the audio to notice that some abnormality may have occurred in the object.

[0130] The conversion data acquisition unit 142 acquires conversion data including elements converted from information on the time series movement of an object in the video data. The conversion data acquisition unit 142 acquires, for example, still image data including still image elements converted from information on the time series movement of an object in the video data. The conversion data acquisition unit 142 acquires, for example, text data including text elements converted from information on the time series movement of an object in the video data. The conversion data acquisition unit 142 acquires, for example, genome data including genome elements converted from information on the time series movement of an object in the video data.

[0131] The video data generating unit 144 generates video data from the converted data acquired by the converted data acquiring unit 142. When the converted data acquiring unit 142 acquires the converted data generated by the data generating unit 118, the video data generating unit 144 generates video data from the converted data using the conversion information stored in the conversion information storage unit 112 and used to generate the converted data. For example, the video data generating unit 144 generates video data 200 from still image data 300 generated by the data generating unit 118. For example, the video data generating unit 144 generates video data 200 from text data 400 generated by the data generating unit 118. For example, the video data generating unit 144 generates video data 200 from genome data 500 generated by the data generating unit 118.

[0132] When acquiring converted data generated by another device, the converted data acquisition unit 142 may also acquire conversion information used by the other device when generating the converted data. The video data generation unit 144 may generate video data from the converted data using the conversion information. For example, the video data generation unit 144 generates video data from still image data generated by another device. For example, the video data generation unit 144 generates video data from text data generated by another device. For example, the video data generation unit 144 generates video data from genome data generated by another device.

[0133] The display output unit 134 may display and output the video data generated by the video data generation unit 144. For example, the display output unit 134 displays and outputs the video data generated by the video data generation unit 144 on a display included in the data processing device 100. For example, the display output unit 134 displays and outputs the video data generated by the video data generation unit 144 on a display included in the communication terminal 30.

[0134] It is not essential that the data processing device 100 includes all of the conversion information storage unit 112, the model storage unit 114, the video data acquisition unit 116, the data generation unit 118, the classification result acquisition unit 120, the model generation unit 122, the output unit 130, the conversion data acquisition unit 142, and the video data generation unit 144. As described above, the data processing device 100 may have a function of generating conversion data from video data, but may not have a function of generating video data from the conversion data. In this case, the data processing device 100 may not include the conversion data acquisition unit 142 and the video data generation unit 144. In addition, the data processing device 100 may have a function of generating video data from conversion data, but may not have a function of generating conversion data from video data. In this case, the data processing device 100 may not include the model storage unit 114, the video data acquisition unit 116, the data generation unit 118, the classification result acquisition unit 120, and the model generation unit 122.

[0135] 9 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the data processing device 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the device according to the present embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0136] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0137] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0138] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0139] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0140] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0141] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0142] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0143] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0144] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0145] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0146] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0147] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0148] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0149] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0150] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order. [Explanation of symbols]

[0151] 20 network, 30 communication terminal, 32 camera, 40 fish farm, 100 data processing device, 102 camera, 112 conversion information storage unit, 114 model storage unit, 116 video data acquisition unit, 118 data generation unit, 120 classification result acquisition unit, 122 model generation unit, 130 output unit, 132 alert output unit, 134 display output unit, 136 audio output unit, 142 conversion data acquisition unit, 144 video data generation unit, 200 video data, 210, 220, 230, 240, 250, 260 video data, 211, 221, 231, 241, 251, 261 frame, 300 still image data, 302, 311, 321, 331, 341, 351, 361 part, 400 Text data, 402, 411, 421, 431, 441, 451, 461 part, 500 genome data, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. a video data acquisition unit that acquires video data of an object; a data generating unit that converts information on time-series movement of the object in the video data into still image elements and generates still image data including the converted still image elements; A data processing device comprising:

2. The data processing device according to claim 1 , wherein the data generating section generates the still image data that represents information on time-series movement of the object in the video data by at least one of pixel values, feature vectors, and edges.

3. The data processing device according to claim 1 , wherein the data generation unit converts information on changes in the positions of the plurality of objects in the video data over time into the elements of the still image, and generates the still image data including the elements of the converted still image.

4. The data processing device according to claim 1 , wherein the data generating section converts information on time-series shape changes of the object in the video data into the elements of the still image, and generates the still image data including the converted elements of the still image.

5. The data processing device according to claim 1 , wherein the data generating section converts information on time-series movement of the object in the video data into text elements, and generates text data including the converted text elements.

6. The data processing device according to claim 1 , wherein the data generation unit converts information on time-series movement of the object in the video data into genome elements, and generates genome data including the converted genome elements.

7. a classification result acquisition unit that inputs the still image data generated by the data generation unit into a still image classification network that has been trained to classify still image data, and acquires a classification result output from the still image classification network; an alert output unit that outputs an alert when the classification result acquired by the classification result acquisition unit satisfies a predetermined condition; A data processing apparatus according to claim 1 , further comprising:

8. a conversion data acquisition unit that acquires still image data including still image elements converted from information on time-series movement of an object in video data; a moving image data generating unit that generates the moving image data from the still image data; The data processing apparatus according to claim 1 , further comprising:

9. a still image data acquisition unit that acquires still image data including still image elements converted from information on time-series movements of objects in video data; a moving image data generating unit that generates the moving image data from the still image data; A data processing device comprising:

10. a video data acquisition unit that acquires video data of an object; a data generating unit that converts information on time-series movement of the object in the video data into text elements and generates text data including the converted text elements; A data processing device comprising:

11. a video data acquisition unit that acquires video data of an object; a data generating unit that converts information on time-series movement of the object in the video data into genome elements and generates genome data including the converted genome elements; A data processing device comprising:

12. A program for causing a computer to function as the data processing device according to any one of claims 1 to 6, 9, 10 and 11.

Images