Information processing device and information processing method

Abstract

To compress a captured image.SOLUTION: An information processing device includes a selection unit that selects an area from a captured video where the appearance of a specified imaging subject satisfies specified conditions, a conversion unit that converts areas other than the area selected by the selection unit to a video quality lower than the specified video quality, and a storage unit that stores the video converted by the conversion unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, there has been provided a non-contact object detection technology using an image captured by a camera (see, for example, Patent Document 1). For example, such object detection technology applies a technology related to artificial intelligence (AI), and for example, a model generated by machine learning or the like is used for object detection.

[0003] In order to use object detection technology, for example, images captured by a camera are stored, and it is desirable to reduce the storage capacity for storing images as much as possible. However, when using object detection technology, if the image quality is reduced too much, the object cannot be detected properly, so it is not possible to uniquely reduce the quality of the entire image.

[0004] Therefore, a method is known in which an arbitrary area is selected in advance, and the selected area is kept at high quality while the outside area is saved at a reduced quality. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2022-524787 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above method has a problem in that information on objects existing outside the selected area is lost. For example, when performing people flow analysis using object detection technology, information on people existing outside the selected area is lost.

[0007] An object of the present invention is to provide an information processing device and an information processing method that compress a captured image so that, for example, a subject to be imaged can be appropriately detected. [Means for solving the problem]

[0008] The information processing device of this embodiment has a selection unit that selects an area from an imaged video in which the appearance of a specified imaging subject satisfies specified conditions, a conversion unit that converts areas other than the area selected by the selection unit to a video quality lower than a specified video quality, and a storage unit that stores the video converted by the conversion unit. Effect of the Invention

[0009] According to the present invention, a captured image can be compressed so that the imaging target can be appropriately detected. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a perspective view illustrating an example of a lighting device according to an embodiment. [Diagram 3] FIG. 3 is a block diagram showing a configuration of a lighting device according to an embodiment. [Figure 4] FIG. 4 is a block diagram showing a configuration of a server device according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of area selection according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of resolution conversion according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of frame rate conversion according to the embodiment. [Figure 8] FIG. 8 is a diagram showing another example of area selection according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the data conversion process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An information processing device (e.g., server device 300; the same applies below) according to an embodiment described below has a selection unit 332 that selects an area from an imaged video in which the aspect of a specific imaging subject satisfies specific conditions, a conversion unit (e.g., data conversion unit 333) that converts areas other than the area selected by the selection unit 332 to a video quality lower than a specific video quality, and a storage unit 335 that stores the video converted by the conversion unit.

[0012] In addition, in the information processing device according to the embodiment described below, when a position where the appearance of the imaging subject satisfies a predetermined condition transitions outside the selected area, the selection unit 332 changes the shape of the area so that the appearance of the imaging subject satisfies the predetermined condition.

[0013] Furthermore, in the information processing device according to the embodiment described below, the selection unit 332 changes the shape of the area so that the area of ​​the area after the change is equivalent to the area of ​​the area before the change.

[0014] Moreover, in the information processing device according to the embodiment described below, when the area of ​​the changed area is larger than the area before the change, the selection unit 332 changes the shape of the area so that the area of ​​the imaging target with the higher priority, which is predetermined for each of the multiple imaging targets, is equal to or smaller than the area before the change.

[0015] Moreover, in the information processing device according to the embodiment described below, the selection unit 332 selects an area in which the aspect of the imaging subject satisfies a predetermined condition at the timing when a captured moving image is acquired.

[0016] Furthermore, in the information processing device according to the embodiment described below, a conversion unit (e.g., data conversion unit 333) extracts a partial image corresponding to the area selected by the selection unit 332 for each image included in a moving image, generates a first image in which the pixel values ​​of the area other than the area are set to a predetermined value, sets the resolution of the target image to a resolution lower than the predetermined resolution, generates a second image in which the pixel values ​​of the area are set to a predetermined value, and combines the first and second images to convert the resolution of the target image.

[0017] [Embodiment] [Information processing system configuration] First, the device configuration and functions of each device of an information processing system 1 according to an embodiment will be described. The device configuration of the information processing system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the information processing system according to an embodiment.

[0018] The information processing system 1 shown in Fig. 1 includes a lighting device 100, a repeater 200, a server device 300, and a terminal device 400. The information processing system 1 may include a plurality of lighting devices 100, a plurality of repeaters 200, a plurality of server devices 300, and a plurality of terminal devices 400. The lighting device 100 is connected to the repeater 200 via a LAN (Local Area Network) or the like so as to be able to communicate with each other by wire or wirelessly. The lighting device 100 communicates with the server device 300 via the repeater 200.

[0019] The lighting device 100 is a lighting device with a camera. For example, the lighting device 100 is a device in which a camera configuration is integrated into a lighting fixture. The lighting device 100 is an edge device equipped with a light source 31 and a camera 21. For example, the lighting device 100 is an information processing device (computer) that executes processing related to capturing an image. Note that the image in this application is a concept that includes a moving image (video) and a still image.

[0020] For example, the lighting device 100 is a lighting device that is installed on the ceiling of a structure such as a factory or a warehouse, and illuminates the internal space of the structure when turned on. FIG. 1 shows a case where the lighting device 100 is installed in the internal space of a factory or a warehouse where an object SB such as luggage is managed. Here, the lighting device 100 will be described with reference to FIG. 2. FIG. 2 is a perspective view showing an example of a lighting device according to an embodiment. The lighting device 100 according to the embodiment is a ceiling-mounted lighting device, which is a so-called base light. In the example shown in FIG. 2, the lighting device 100 has a camera unit 20 and a lighting unit 30. Note that the lighting device 100 is not limited to a base light, and may be a ceiling light, a downlight, a spotlight, or the like.

[0021] Camera unit 20 is an imaging device that captures an image of a predetermined area around lighting device 100. In Fig. 1, camera unit 20 captures an image of an imaging range AR that corresponds to the angle of view of camera 21. For example, imaging range AR indicates the imaging range of camera 21. For example, camera unit 20 captures an image, such as a moving image such as a video or a still image, targeting the imaging range AR.

[0022] For example, when an object SB is located within the shooting range AR, the camera unit 20 captures an image including the object SB. Also, when an entity (also referred to as a "carrying entity") carrying an object SB such as luggage managed in the space is located within the shooting range AR, the camera unit 20 captures an image including the carrying entity (also referred to as a "carrier" or "moving object"). For example, an object SB etc. is a transported target (moved object) in comparison with a moving object (carrying entity). For example, when a person HM, who is an example of a carrying entity (also referred to as a "person"), is located within the shooting range AR, the camera unit 20 captures an image including the person HM.

[0023] The transporting subject is not limited to a person, but may include various equipment (moving bodies) as long as it is capable of transporting the object SB. For example, when a forklift FL, which is an example of a transporting subject, is located within the shooting range AR, the camera unit 20 captures an image including the forklift FL. Note that, although a forklift is illustrated as an example of a transporting subject other than a person in FIG. 1, the transporting subject other than a person may be various equipment (moving bodies) such as an automatic guided vehicle, an autonomous mobile robot, a cart, etc. In other words, the transporting subject is not limited to a person, but may be various equipment as long as it is capable of transporting the object SB, i.e., the position of the object SB can be changed.

[0024] The repeater 200 is a device having a function of performing relay processing. For example, the repeater 200 is a so-called gateway that connects a network within a facility in which the lighting device 100 is installed to an external network such as the Internet.

[0025] The server device 300 is an information processing device that executes various information processes described later. The server device 300 is a device used as a server of a client-server system. For example, the server device 300 is an information processing device (computer) that functions as a so-called cloud server. The server device 300 compresses images captured by the camera 21 so that they can be used for people flow analysis that analyzes the flow of people. The server device 300 also executes object detection processing to detect an imaging target from the images captured by the camera 21. The server device 300 also executes processing to detect a carrying subject such as a person from the images captured by the camera 21. That is, the server device 300 controls various detection processes and executes processes based on the results of various detections, which will be described in detail later.

[0026] Server device 300 is connected to relay 200 and terminal device 400 via a predetermined communication network such as the Internet, either wired or wirelessly. Server device 300 may be connected to relay 200 and terminal device 400 in any manner as long as information can be transmitted and received, and may be connected to relay 200 and terminal device 400 via wired or wireless communication. For example, server device 300 may communicate with terminal device 400 via any device (such as a gateway device).

[0027] The terminal device 400 is a device having a function of displaying information. For example, the terminal device 400 is an information processing device (computer) used by a user. For example, the terminal device 400 is a mobile terminal device that can communicate with any server device 300 via a predetermined network. FIG. 1 shows a case where the terminal device 400 is a notebook PC (Personal Computer). Note that the terminal device 400 may be not only a notebook PC, but also a smart device such as a smartphone or a tablet, or a computer such as a desktop PC.

[0028] The terminal device 400 displays information provided by the server device 300. For example, the terminal device 400 transmits information requesting information (request information) to the server device 300, and displays the information received from the server device 300. For example, the terminal device 400 transmits request information specifying information to the server device 300, and displays the information received from the server device 300. For example, the terminal device 400 displays content including moving images (video) compressed by the server device 300 to perform people flow analysis.

[0029] [Lighting equipment configuration example] Next, the configuration of the lighting device 100 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the lighting device according to the embodiment. For example, the lighting device 100 is an information processing device (computer) that functions as a lighting device with a camera. As shown in Fig. 3, the lighting device 100 has a communication unit 10, a camera unit 20, a lighting unit 30, a control unit 40, and a storage unit 50.

[0030] The communication unit 10 transmits and receives information to and from other devices such as the repeater 200. The communication unit 10 is realized, for example, by a predetermined communication circuit or a NIC (Network Interface Card). The communication method by the communication unit 10 may be, for example, wireless communication such as a local area network (LAN). Note that the communication method by the communication unit 10 may be any communication method, for example, wired communication, infrared communication, etc.

[0031] Camera unit 20 has a camera 21, a camera control unit 22, and a storage unit 23. Camera 21 has an imaging element that electronically acquires an image, such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and captures an image corresponding to a predetermined angle of view. For example, camera 21 has a fixed imaging range (imaging range AR in FIG. 1), and captures an image of the fixed imaging range.

[0032] The camera control unit 22 controls the photographing by the camera 21. In addition, the camera control unit 22 outputs the image photographed by the camera 21 to the control unit 40.

[0033] Storage unit 23 stores programs and data for implementing various controls of camera unit 20. Storage unit 23 may store images captured by camera 21 or images (image data) captured by camera 21. Storage unit 23 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0034] The lighting unit 30 includes a light source 31, a lighting control unit 32, and a storage unit 33. The light source 31 includes a semiconductor light emitting element such as an LED (Light Emitting Diode). The light source 31 is turned on or off in a manner corresponding to a control signal output from the lighting control unit 32.

[0035] The lighting control unit 32 controls turning on and off of the light sources 31. The lighting control unit 32 turns on the light sources 31 that are to be turned off, or turns off the light sources 31 that are to be turned on. In the case of the lighting unit 30 in which the light source 31 can be dimmed, the lighting control unit 32 may control the dimming degree of the light source 31.

[0036] The storage unit 33 stores programs and data for implementing various controls of the lighting unit 30. The storage unit 33 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 33 may be provided in the storage unit 50 described later.

[0037] The storage unit 50 stores various information related to information processing. The storage unit 50 stores programs for implementing various controls of the control unit 40. The storage unit 50 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 50 may store information stored in the storage unit 23 or information stored in the storage unit 33.

[0038] The control unit 40 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing various programs stored in an internal memory using a RAM as a working area. The control unit 40 can also be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 40 has an acquisition unit 41, a processing unit 42, and a transmission unit 43, and realizes or executes the functions and actions of information processing described below. The internal configuration of the control unit 40 is not limited to the configuration shown in FIG. 3, and may be other configurations as long as they perform information processing described below.

[0039] The acquisition unit 41 acquires various information. The acquisition unit 41 acquires various information from the camera unit 20. The acquisition unit 41 acquires images captured by the camera unit 20 from the camera unit 20. The acquisition unit 41 acquires various information from the lighting unit 30. The acquisition unit 41 acquires various information from the memory unit 50. The acquisition unit 41 acquires various information from an external information processing device. The acquisition unit 41 receives various information from an external information processing device via the communication unit 10. The acquisition unit 41 receives information from the repeater 200. The acquisition unit 41 receives information from the server device 300 via the repeater 200.

[0040] The processing unit 42 processes the images captured by the camera unit 20 and executes a predetermined process. The processing unit 42 performs image processing. The processing unit 42 stores various information in the storage unit 50. Here, the processing unit 42 stores the images captured by the camera 21 in a manner according to settings from the terminal device 400. For example, the processing unit 42 may store only one week's worth of images in the storage unit 50 and erase images that have been captured more than one week ago. Furthermore, when a setting is made to upload images to the cloud, the processing unit 42 may upload the captured images to the server device 300 without registering them in the storage unit 50.

[0041] The transmitting unit 43 transmits various information to an external device via the communication unit 10. The transmitting unit 43 transmits various information to the repeater 200. The transmitting unit 43 transmits an image captured by the camera unit 20 to an external device. The transmitting unit 43 transmits the image to the server device 300 via the repeater 200.

[0042] [Server device configuration example] Next, the configuration of the server device 300 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the server device according to the embodiment. As shown in Fig. 4, the server device 300 has a communication unit 310, a storage unit 320, and a control unit 330.

[0043] The communication unit 310 is realized by, for example, a predetermined communication circuit or a NIC. For example, the communication unit 310 can communicate with external devices such as the repeater 200 and the terminal device 400. For example, the communication unit 310 can communicate with the terminal device 400 used by the administrator of the information processing system 1.

[0044] The storage unit 320 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 320 stores various information related to information processing. The storage unit 320 functions as a storage unit that stores images such as moving images (video) or still images captured by the camera 21. That is, the storage unit 320 stores images captured by the camera 21. The storage unit 320 stores images captured by the camera 21 with a fixed shooting range. For example, the storage unit 320 according to the embodiment has an image information storage unit 321 and a processing information storage unit 322.

[0045] The storage unit 320 may store various information according to the purpose, not limited to the above. For example, the storage unit 320 stores various information received from the lighting device 100, the repeater 200, etc. in addition to the above.

[0046] The image information storage unit 321 stores video after compressing moving images (images) captured by the lighting device 100. The image information storage unit 321 may also temporarily store information related to moving images captured by the lighting device 100. The image information storage unit 321 may also store not only images themselves such as moving images and still images, but also file path names indicating storage locations of these images. The image information storage unit 321 is not limited to the above, and may store various types of information according to the purpose.

[0047] The processing information storage unit 322 stores information used for various information processes. For example, the processing information storage unit 322 stores information indicating input items used for data conversion processing of moving images captured by the lighting device 100. The data conversion processing refers to conversion processing for compressing moving images.

[0048] As an example, the input items include area, area size, area shape, post-conversion resolution, and post-conversion frame rate. In the case of performing people flow analysis, for example, the area refers to an area where people flow is expected. Such an area has a higher resolution than the outside of the area in each frame of the video. The resolution of such an area is, for example, a resolution that indicates the quality required for people flow analysis. For example, in the case of performing people flow analysis, 1280 x 720 is required for wide-angle, and 1280 x 1280 is required for omnidirectional. On the other hand, the area other than the area in each frame is set to a resolution lower than a predetermined resolution. In addition, the area size referred to here is the area of ​​the above-mentioned area. In addition, the area shape referred to here refers to the shape of the newly added partial area. The area shape may be a circle or a rectangle, and may be determined in advance. In addition, the post-conversion resolution referred to here means the compression rate at the post-conversion resolution. In the case of performing people flow analysis, for example, 5 fps (frames per second) is sufficient. The processing information storage unit 323 is not limited to the above, and may store various types of information depending on the purpose.

[0049] The control unit 330 is realized, for example, by a CPU, an MPU, or the like, executing various programs stored in an internal memory using a RAM as a working area. The control unit 330 can also be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 330 has an acquisition unit 331, a selection unit 332, a data conversion unit 333, a synthesis unit 334, and a storage unit 335, and realizes or executes the functions and actions of information processing described below. The internal configuration of the control unit 330 is not limited to the configuration shown in FIG. 4, and may be other configurations as long as they perform information processing described below.

[0050] The acquisition unit 331 acquires various types of information. For example, the acquisition unit 331 acquires a moving image captured by the lighting device 100.

[0051] The selection unit 332 selects an area where the aspect of a predetermined imaging target satisfies a predetermined condition from the moving image captured by the lighting device 100. For example, the selection unit 332 divides the moving image acquired by the acquisition unit 331 into frame units. Then, the selection unit 332 performs the following process for each frame. For a certain frame, the selection unit 332 selects an area where the aspect of a predetermined imaging target satisfies a predetermined condition by the area size of the input item. The process of detecting the part where the aspect of the imaging target satisfies the predetermined condition may use, for example, AI (Artificial Intelligence) of a moving object that applies a people flow analysis function. The AI ​​of the moving object may include, for example, a detection function of a moving object. That is, the selection unit 332 may specify the position of the imaging target using such a process, and select an area where the aspect of the imaging target at the specified position satisfies the predetermined condition. Note that the area where the imaging target satisfies the predetermined condition may be selected using any conventional technology process, or may be selected manually.

[0052] Here, the imaging target may be, for example, a person or an object, and may be any object that moves. Examples of objects include moving objects such as a forklift. The predetermined condition may be, for example, a condition that the density of the imaging target is higher than a predetermined degree. The predetermined condition may be, for example, a condition that the moving speed of the imaging target is higher than a predetermined moving speed. The predetermined condition may be the imaging target itself that is of interest. Note that, although the imaging target and the predetermined condition are not limited thereto, in the embodiment, the imaging target is described as a person, and the predetermined condition is a condition that the density of people is higher than a predetermined degree. That is, the selection unit 332 selects an area that satisfies the condition that the density of people is higher than a predetermined degree from the moving image captured by the lighting device 100.

[0053] Furthermore, the selection unit 332 changes the shape of the area so that the aspect of the predetermined imaging target satisfies a predetermined condition. For example, when the position where the aspect of the imaging target satisfies the predetermined condition transitions outside the area for a frame following the frame in which the area is selected, the selection unit 332 changes the shape of the area so that the area of ​​the changed area becomes equivalent to the area size of the input item. That is, the selection unit 332 adds a partial area that satisfies the predetermined condition and changes the shape of the area so that the area of ​​the added partial area is deleted from the area. The shape of the newly added partial area may be the area shape of the input item.

[0054] Furthermore, the selection unit 332 holds the position and shape of the selected area for each frame as an input item.

[0055] There may be a plurality of modes of the imaging target that satisfy the predetermined condition. In such a case, the selection unit 332 changes the shape of the area for each mode of the imaging target that satisfies the predetermined condition so that the area of ​​the changed area is equivalent to the area size of the input item. Here, the selection unit 332 may consider a case where the area of ​​the changed area is larger than the area size of the input item. In such a case, for example, the imaging target to be followed may be ranked in advance in terms of priority. Then, the selection unit 332 changes the shape of the area in order from the imaging target with the highest priority so that it is equal to or smaller than the area size of the input item, and if the area of ​​the changed area is larger than the area size of the input item, the change in the shape of the area may be stopped. In addition, if the predetermined condition is, for example, a condition that the moving speed of the imaging target is faster than a predetermined moving speed, the ranking may be performed so that the higher the moving speed, the higher the priority.

[0056] The data conversion unit 333 converts the moving images captured by the lighting device 100 into images with a resolution lower than a predetermined resolution for areas other than the area selected by the selection unit 332. Note that the moving images captured by the lighting device 100 are assumed to be high quality images required for people flow analysis.

[0057] For example, the data conversion unit 333 divides the moving image acquired by the acquisition unit 331 into frame units. The data conversion unit 333 performs the following process for each divided frame. The data conversion unit 333 extracts an image corresponding to the area saved in the input item from the corresponding frame, and generates the extracted image. The data conversion unit 333 then sets all RGB values ​​of pixels outside the area for the generated image to a predetermined value. The predetermined value is, for example, "0." In other words, the RGB of pixels outside the area are set to, for example, (0,0,0). This generates a "high-quality extracted image" in which the area is a high-quality image.

[0058] Then, the data conversion unit 333 sets the entire image to a low resolution for the corresponding frame. As an example, the data conversion unit 333 generates an image in which the entire image is set to a low resolution by taking the average of RGB of adjacent pixels according to the changed resolution (resolution compression rate) stored in the input item. Then, the data conversion unit 333 sets the RGB values ​​of pixels of the image included in the generated image and corresponding to the area stored in the input item to a predetermined value. That is, the RGB of pixels within the area are set to, for example, (0,0,0). As a result, a "low-quality image" is generated in which the image outside the area is a low-quality image.

[0059] Then, the two generated images for the corresponding frame are combined to generate a complete image. This allows the ratio of the total number of pixels contained within the area to the total number of pixels in the entire image of the frame to contribute to the final degree of data compression. As a result, the data conversion unit 333 can reduce the data volume of the captured video without losing any necessary information by converting the resolution of the images that make up the video.

[0060] In the above description, the data conversion unit 333 compresses the moving image by converting the resolution of the images constituting the moving image, but the moving image may be compressed by converting the frame rate of the moving image in addition to converting the resolution. The following describes the data conversion process for compressing the moving image by converting the frame rate of the moving image.

[0061] The data conversion unit 333 calls an image (low-quality original image) that is a starting point among the low-quality images of each frame obtained by the resolution conversion. The low-quality original image to be called may be, for example, the first image of the video and the first image in which the shape of the area has changed. In addition, the low-quality original image to be called may be, for example, the first image of the video and an image in which even one pixel has changed among the low-quality images. In addition, the low-quality original image to be called may be, for example, the first image of the video and an image in which the amount of change has exceeded a predetermined threshold among the low-quality images. In addition, the low-quality original image to be called may be, for example, the first image of the video and an image in which a predetermined time has passed among the low-quality images.

[0062] Then, the data conversion unit 333 calls up the high-quality extracted images in all frames obtained by the resolution conversion.

[0063] Then, the data conversion unit 333 selects frames according to the converted frame rate (frame rate compression ratio) of the input item.

[0064] The synthesis unit 334 synthesizes a low-quality image and a high-quality extracted image for each frame. For example, the synthesis unit 334 synthesizes a low-quality image in a frame selected by the data conversion unit 333 and a high-quality extracted image. That is, the synthesis unit 334 generates a completed image (frame) by taking the sum of RGB of each pixel of the two generated images. At this time, the synthesis unit 334 synthesizes high-quality extracted images in different frames with one low-quality original image as a starting point until the low-quality original image changes, and generates a completed image. As a result, regarding the resolution and frame rate, the ratio of the resolution and frame rate after conversion to the resolution and frame rate at the time of shooting as a reference contributes to the final degree of data compression. As a result, the synthesis unit 334 can reduce the data capacity of the captured moving image without losing necessary information by resolution conversion and frame conversion of the images constituting the moving image.

[0065] The storage unit 335 regenerates a moving image (video) from each of the synthesized complete images, and stores the regenerated moving image (video) in the image information storage unit 321. Although the storage unit 335 has been described as storing the generated moving image (video) in the image information storage unit 321, the present invention is not limited to this, and the generated moving image (video) may be stored in a predetermined HDD (Hard Disk Drive) or cloud. This allows the storage unit 335 to reduce the file size of the captured moving image without losing necessary information. In other words, the storage unit 335 can stably reduce the file size to a constant amount so that the imaging target can be appropriately detected.

[0066] [Example of area selection] Here, the area selection performed by the selection unit 332 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of area selection according to an embodiment. In Fig. 5, the selection unit 332 selects an area that satisfies the condition that the density of people is higher than a predetermined level from a captured video. For convenience of explanation, a case in which areas are selected for three frames A1, A2, and A3 will be described here.

[0067] In the left diagram of FIG. 5, the selection unit 332 selects an area that satisfies the condition that the density of people is higher than a predetermined level for the earliest frame A1 of the three frames. Here, a cross-shaped area that is generated along the passage and is an area that targets people walking along the passage is selected as the area a0 that satisfies the condition. The selected area a0 may be selected, for example, using AI that applies a people flow analysis function, or may be selected manually. The area a0 is selected by the area size saved in the input item. Then, the selection unit 332 saves the position and shape of the selected area for frame A1 in the input item.

[0068] In the middle diagram of FIG. 5, the selection unit 332 selects an area that satisfies the condition that the density of people is higher than a predetermined level for frame A2, which is the next frame of the earliest frame A1 among the three frames. Here, a person is about to deviate from the passage, but has not yet deviated. The same area a0 as the area selected in frame A1 is selected. The selected area a0 may be selected, for example, as follows. The selection unit 332 may use an AI that applies a people flow analysis function to identify the position of the imaging target, and determine whether the identified position is outside the range of area a0, thereby determining whether to leave the shape of area a0 as it is or change it. Then, the selection unit 332 holds the position and shape of the selected area for frame A2 as an input item.

[0069] In the right diagram of FIG. 5, the selection unit 332 selects an area that satisfies the condition that the density of people is higher than a predetermined level for the frame A3 next to the frame A2 out of the three frames. Here, it is a case where a person has deviated (left) the passage. When the position that satisfies the condition that the density of people is higher than a predetermined level transitions outside the area, the selection unit 332 changes the shape of the area so that the area of ​​the changed area is equivalent to the area size saved in the input item. That is, the selection unit 332 adds a partial area that satisfies the condition and changes the shape of the area so that the area of ​​the added partial area is deleted from the area. It is assumed that the position that satisfies the condition that the density of people is higher than a predetermined level transitions to a partial area a31 outside the range of the area a0. Then, the partial area a31 that satisfies the condition is added. The shape of the newly added partial area a31 may be the area shape of the input item (here, for example, a circle). Then, the partial area d31 in the area a0 is deleted so that the area of ​​the added partial area a31 is deleted from the area a0. The area whose shape has been changed is the region indicated by the symbol a3. In other words, if the position that satisfies the condition that the density of people is higher than a predetermined level is outside area a0, the shape of the selected area is changed to track the relevant person. Then, the selection unit 332 stores the position and shape of the selected (changed) area in the input items for frame A3. This allows the selection unit 332 to reduce the capacity of the final file to a certain amount by making the areas of the selected areas equivalent.

[0070] [An example of resolution conversion] Here, the resolution conversion performed by the data conversion unit 333 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of resolution conversion according to an embodiment. In Fig. 6, the data conversion unit 333 converts areas of images (frames) constituting a captured moving image other than the area selected by the selection unit 332 to a resolution lower than a predetermined resolution. For convenience of explanation, it is assumed here that the areas of the three frames A1, A2, and A3 shown in Fig. 5 are selected. It is assumed that the three frames A1, A2, and A3 are high-quality images required for people flow analysis.

[0071] The data conversion unit 333 extracts an image corresponding to the area stored in the input item from the frame A1, and generates the extracted image. The extracted image is a cross-shaped area. The data conversion unit 333 then sets the RGB values ​​of the pixels outside the generated image to (0,0,0) all to (0,0,0) (see symbol b1), which should be lower than the resolution of the area a0. This generates a "high-quality extracted image" in which the area is a high-quality image (middle row of A1).

[0072] The data conversion unit 333 generates an image in which the entire image is set to a low resolution by taking the average of the RGB of adjacent pixels according to the changed resolution (resolution compression ratio) stored in the input item (see symbol b2).Then, the data conversion unit 333 sets the RGB values ​​of pixels in the generated image that correspond to the area stored in the input item to (0,0,0).As a result, a "low-quality image" is generated in which the image outside the area is a low-quality image (top row of A1).

[0073] Thereafter, for frame A1, the two generated images (the low-quality image and the high-quality extracted image) are synthesized to generate a completed image.

[0074] Similarly, for frame A2, the data conversion unit 333 performs resolution conversion in the same manner as for frame A1.

[0075] For frame A3, the data conversion unit 333 extracts an image (a3) ​​corresponding to the area saved in the input items from frame A3, and generates the extracted image. Then, the data conversion unit 333 sets the RGB values ​​of the pixels outside the generated image to (0,0,0) (see symbol b3), which should be lower than the resolution of area a3. This generates a "high-quality extracted image" in which the area is a high-quality image (middle row of A3).

[0076] The data conversion unit 333 generates an image in which the entire image is set to a low resolution by taking the average of the RGB of adjacent pixels according to the changed resolution (resolution compression ratio) stored in the input items (see symbol b4).Then, the data conversion unit 333 sets the RGB values ​​of pixels of the image (a3) ​​included in the generated image and corresponding to the area stored in the input items to (0,0,0).As a result, a "low-quality image" is generated in which the image outside the area is a low-quality image (top row of A3).

[0077] Thereafter, for frame A3, the two generated images (the low-quality image and the high-quality extracted image) are synthesized to generate a completed image.

[0078] [An example of frame rate conversion] Next, the frame rate conversion performed by the data conversion unit 333 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of frame rate conversion according to the embodiment. For convenience of explanation, a case where the frame rates of three frames A1, A2, and A3 shown in Fig. 6 are converted will be described here. In Fig. 7, the converted frame rate (frame rate compression ratio) saved in the input item is assumed to be 3 fps.

[0079] The data conversion unit 333 calls an image (low-quality original image) that is the starting point of the low-quality image in the frame obtained by resolution conversion. Here, the data conversion unit 333 calls the low-quality image in frame A1 as the low-quality original image. This is because the low-quality image in frame A1 is the first image in the video. In addition, the data conversion unit 333 calls the low-quality image in frame A3 as the low-quality original image. This is because the low-quality image in frame A3 is the first image in which the shape of the area has changed.

[0080] Then, the data conversion unit 333 calls up the high-quality extracted images in all of the frames A1, A2, and A3 obtained by the resolution conversion. The data conversion unit 333 selects frames according to the converted frame rate (frame rate compression ratio) of the input item.

[0081] Then, the synthesis unit 334 synthesizes high-quality extracted images in different frames with one low-quality original image until the low-quality original image changes, and generates a completed image. Here, the synthesis unit 334 synthesizes the high-quality extracted image of frame A1 with one low-quality original image of frame A1 until the low-quality original image of frame A3 is obtained, and synthesizes the high-quality extracted image of frame A2. Then, the synthesis unit 334 synthesizes the high-quality extracted image of frame A3 with one low-quality original image of frame A3. In other words, since the post-conversion frame rate (frame rate compression rate) is 3 fps, the synthesis unit 334 synthesizes the same low-quality original image with two high-quality extracted images, and generates two completed images. Then, since the shape of the area is different for the remaining one frame, the synthesis unit 334 changes the low-quality image that is the base, and synthesizes the low-quality original image with the high-quality extracted image.

[0082] When the resolution and frame rate conversion process is completed, the storage unit 335 regenerates the image from each of the synthesized frames, and stores the regenerated image in the image information storage unit 321. Note that the storage unit 335 may store the image in a HDD or in the cloud as appropriate.

[0083] [Another example of area selection] Here, another example of area selection performed by the selection unit 332 will be described with reference to Fig. 8. Fig. 8 is a diagram showing another example of area selection according to the embodiment. Fig. 8 describes an example of area selection when there are multiple positions outside the area that satisfy the condition that the human density is higher than a predetermined degree. It should be noted that the area set first for the input item is assumed to be the cross-shaped area a0 shown in the left diagram of Fig. 5.

[0084] As shown in FIG. 8, when a person who satisfies the condition enters the circular lattice portion b2 from the area a0 in addition to the circular lattice portion b1, the selection unit 332 changes the shape of the area so that the area of ​​the changed area is equivalent to the area size saved in the input items. That is, the selection unit 332 changes the shape of the area a0 so that the circular lattice portion b1 and the circular slant portion b2 that satisfy the condition are added and the area of ​​the added partial area is deleted from the area a0. Here, the lattice portion d1 included in the area a0 is deleted due to the addition of the lattice portion b1. The slant portion d2 included in the area a0 is deleted due to the addition of the slant portion b2. The area whose shape has been changed is the region indicated by the symbol b3. That is, when a plurality of positions that satisfy the condition that the density of people is higher than a predetermined degree are out of the area a0, the shape of the selected area is remade in a manner that follows the corresponding plurality of people.

[0085] [Data conversion process flow] Next, a description will be given of the flow of data conversion processing in the server device 300. Fig. 9 is a flowchart showing an example of the data conversion processing according to the embodiment.

[0086] 9, the server device 300 acquires captured data (step S11). For example, the server device 300 acquires video (moving images) captured by the camera 21.

[0087] Then, the server device 300 acquires the input items (step S12). The input items include the area, the area size, the area shape, the post-conversion resolution, and the post-conversion frame rate. In Fig. 9, the area of ​​the first frame is assumed to be set in advance.

[0088] Then, for the second and subsequent frames, the server device 300 executes a detection process of the imaging target by applying the people flow analysis function (step S13). For example, the server device 300 detects the position of the imaging target by applying the people flow analysis function. Then, the server device 300 determines whether the detected imaging target is outside the area (step S14). If it is determined that the detected imaging target is not outside the area (step S14; No), the server device 300 proceeds to step S16 to select the next frame.

[0089] On the other hand, if it is determined that the detected imaging target is outside the area (step S14; Yes), the server device 300 changes the area of ​​the target frame (step S15). For example, the server device 300 changes the shape of the area so that the area of ​​the changed area is equivalent to the area size saved in the input items. The shape of the newly added partial area may be the area shape saved in the input items. Then, the server device 300 proceeds to step S16 to select the next frame.

[0090] In step S16, the server device 300 determines whether or not there is a next frame (step S16). If it is determined that there is a next frame (step S16; Yes), the server device 300 selects the next frame and proceeds to step S13.

[0091] On the other hand, if it is determined that there is no next frame (step S16; No), the server device 300 extracts an image within the area for each frame and masks the outside of the area (step S17). The generated image is a high-quality extracted image shown in the middle of FIG. 6 and is represented by the symbol G1. Then, the server device 300 generates an image of the outside of the area for each frame with the inside of the area masked (step S18). The generated image of the outside of the area is represented by the symbol G2.

[0092] Then, the server device 300 judges whether or not there is a next frame (step S19). If it is judged that there is a next frame (step S19; Yes), the server device 300 selects the next frame and proceeds to step S17.

[0093] On the other hand, if it is determined that there is no next frame (step S19; No), the server device 300 processes the frame rate of the image G2 outside the area for each frame to an arbitrary low quality (step S20). For example, the server device 300 processes the image G2 outside the area to a low quality by selecting it according to the converted frame rate stored in the input item.

[0094] Then, for each frame whose frame rate has been processed to an arbitrary low quality, the server device 300 processes the resolution of the image G2 outside the area to an arbitrary quality (step S21). For example, the server device 300 processes the resolution of the image G2 outside the area to a low quality according to the changed resolution stored in the input item. The generated image outside the area is represented as G2'.

[0095] Then, the server device 300 composites the outside-area image G2' with the high-quality extracted image G1 (step S22). For example, the server device 300 composites the outside-area image G2' with the high-quality extracted image G1 in a frame selected according to the converted frame rate. At this time, the server device 300 composites the single outside-area image G2' as a starting point with the high-quality extracted image G1 in a different frame until the outside-area image G2' changes, thereby generating a completed image.

[0096] Then, the server device 300 regenerates data (video) from each completed image, and stores the regenerated data (video) in a HDD, a cloud, etc. (Step S23). Then, the server device 300 ends the data conversion process.

[0097] In the above flowchart, in step S17, the server device 300 extracts an image within the area for each frame, masks the outside of the area, and generates a high-quality extracted image G1. Then, the server device 300 generates an image outside the area G2 with the inside of the area masked for each frame. Then, in step S20, the server device 300 processes the frame rate of the image outside the area G2 for each frame to an arbitrary low quality, and then in step S21, processes the resolution of the image outside the area G2 to a low quality. However, the server device 300 is not limited to this, and may extract an image within the area for each frame, mask the outside of the area, and generate a high-quality extracted image G1. Then, the server device 300 may process the resolution of the image for each frame to a low quality and mask the inside of the area to generate an image for low quality. In this case, since the image for low quality has already been generated, in step S21, the server device 300 only needs to process the frame rate of the image outside the area G2 for each frame to an arbitrary low quality. That is, the server device 300 does not need to process the image outside the area at a low resolution in step S21.

[0098] As described above, the server device 300 selects an area from the captured video where the aspect of a predetermined imaging target satisfies a predetermined condition. The server device 300 converts the quality of the video for areas other than the selected area to a lower quality than the predetermined video quality. The server device 300 stores the converted video. This allows the server device 300 to compress the captured image so that the imaging target can be appropriately detected.

[0099] [Other system configuration examples] The above-described configuration of the information processing system 1 is merely an example, and the information processing system 1 may have any device configuration as long as the desired processing is possible. For example, the information processing system 1 may not have the repeater 200. In addition, the various processes performed by the server device 300 may be executed by any device included in the information processing system 1, not limited to the server device 300. For example, at least a part of the various processes performed by the server device 300 (e.g., the selection unit 332, the data conversion unit 333, the synthesis unit 334, etc.) may be executed by the repeater 200 or the lighting device 100. In this way, the various processes performed by the server device 300 may be distributed and processed by a plurality of devices included in the information processing system 1. For example, the lighting device 100 executes the acquisition unit 331 to the synthesis unit 334, and the server device 300 executes the storage unit 335, so that not only the data volume of the moving image can be compressed, but also the communication volume from the lighting device 100 to the server device 300 can be compressed.

[0100] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as well as the scope and gist of the invention. In addition, these embodiments and their modifications can be appropriately combined as long as the processing contents are not contradictory. [Explanation of symbols]

[0101] 1. Information Processing Systems 100 Lighting Equipment 10. Communications Department 20 Camera unit 21 Camera 30 Lighting Unit 31 Light source 40 Control section 41 Acquisition Department 42 Processing section 43 Transmitter 50 Storage section 200 Repeater 300 Server device (information processing device) 310 Communications Department 320 Storage section 321 Image information storage unit 322 Processing information storage unit 330 Control Unit 331 Acquisition Department 332 Selection section 333 Data Conversion Department 334 Synthesis Section 335 Preservation Department 400 Terminal Equipment

Claims

1. a selection unit that selects an area in which a state of a predetermined imaging target satisfies a predetermined condition from the captured moving image; a conversion unit that converts areas other than the area selected by the selection unit into a video quality lower than a predetermined video quality; a storage unit for storing the moving image converted by the conversion unit; An information processing device having the above configuration.

2. When a position where the state of the imaging target satisfies a predetermined condition is transitioned to outside the selected area, the selection unit changes a shape of the area so that the state of the imaging target satisfies the predetermined condition.

2. The information processing apparatus according to claim 1,

3. The selection unit changes the shape of the area so that the area after the change is equivalent to the area before the change.

2. The information processing apparatus according to claim 1,

4. When the area of ​​the area after the change is larger than the area before the change, the selection unit changes the shape of the area so that the area of ​​the imaging target having a higher priority than the area before the change is smaller than the area before the change.

4. The information processing apparatus according to claim 3.

5. The selection unit selects the area in which the state of the imaging subject satisfies a predetermined condition at a timing when the captured moving image is acquired.

2. The information processing apparatus according to claim 1,

6. The conversion unit extracts a partial image corresponding to the area selected by the selection unit for each image included in the moving image, generates a first image in which pixel values ​​other than the area are set to a predetermined value, sets the resolution of the target image to a resolution lower than the predetermined resolution, generates a second image in which pixel values ​​of the area are set to a predetermined value, and combines the first image and the second image to convert the resolution of the target image.

2. The information processing apparatus according to claim 1,

7. a selection step of selecting an area in which a state of a predetermined imaging target satisfies a predetermined condition from the captured moving image; a conversion step of converting areas other than the area selected by the selection step into a quality lower than a predetermined moving image quality; a storing step of storing the moving image converted by the converting step; An information processing method comprising:

Images