Data collection device and video system

Abstract

This data collection device collects image data and is composed of a computer including: a calculation device that executes predetermined calculation processing; and a storage device that can be accessed by the calculation device. The calculation device is provided with a video determination unit that outputs data in which image data is labeled in accordance with a parameter of an image capture environment. The video determination unit outputs data in which image data requiring enhancement is labeled as incorrect data and image data not requiring enhancement is labeled as correct data in accordance with the image capture environment.

Description

Data collection equipment and imaging system Incorporation by Reference

[0001] This application claims priority from Japanese Patent Application No. 2024-36139, filed on March 8, 2024, the contents of which are incorporated herein by reference.

[0002] The present invention relates to a data collection device that labels learning images for learning a sharpening process suited to the installation environment.

[0003] Conventional camera units must capture images in a variety of environments, so the sharpening intensity and photometry area are set to fixed standards. However, in harsh environments such as dark nighttime or backlit conditions, the image level can be low or saturated, making it difficult to obtain sufficient images. On the other hand, if the sharpening intensity and photometry area are set to match images with low or saturated image levels, unnatural images can be captured when monitoring normal images. For this reason, in order to collect learning data for environment-specific sharpening, it was necessary to record images captured over a certain period of time and manually label the learning data from the recorded images.

[0004] The following prior art exists as background art in this technical field: Patent Document 1 (JP 2020-113844 A) describes a camera device that is connected to a server device so as to be able to communicate with the server device, and that includes an imaging unit that captures an outdoor area to be monitored, a detection unit that detects an object that appears in the captured image of the area to be monitored and generates a cutout image of the object, an image correction unit that uses the cutout image of the object and an image correction model to generate a sharpened image that is used in a matching process of the object on the server device, and a communication unit that associates the cutout image of the object and the sharpened image with identification information of the object and sends them to the server device.

[0005] In conventional methods, training data for learning sharpening processing tailored to the environment is extracted manually, which requires a lot of time. Also, if emphasis is placed on the naturalness of images under normal circumstances, the sharpening effect may not be fully realized.

[0006] The present invention aims to provide a data collection device that labels learning images for learning sharpening processing suited to the installation environment.

[0007] A representative example of the invention disclosed in the present application is as follows: That is, a data collection device for collecting image data is configured by a computer having an arithmetic unit that executes predetermined arithmetic processing and a storage device accessible by the arithmetic unit, the arithmetic unit includes an image determination unit that outputs data in which image data is labeled according to parameters of an image shooting environment, and the image determination unit outputs data in which image data that requires sharpening is labeled as incorrect data and image data that does not require sharpening is labeled as correct data according to the image shooting environment.

[0008] In addition, in the data collection device according to one example of the present invention, the parameter of the photographing environment is at least one of weather, position of the sun, and age of the moon.

[0009] Furthermore, one example of a video system according to the present invention is a video system that collects image data, and includes a camera that captures video and outputs image data, and a data collection device that collects the image data captured by the camera, wherein the camera has a sharpening processing unit that sharpens the captured image data in accordance with the shooting environment of the image, and the data collection device has a video assessment unit that outputs data in which image data is labeled in accordance with parameters of the shooting environment of the image, and the video assessment unit labels image data that requires sharpening as incorrect data and outputs data in which image data that does not require sharpening is labeled as correct data in accordance with the shooting environment.

[0010] In addition, in the video system according to one example of the present invention, the parameter of the shooting environment is at least one of weather, position of the sun, and age of the moon.

[0011] According to one aspect of the present invention, it is possible to reduce the time required to extract learning data for learning a sharpening process suited to the shooting environment. Furthermore, clear images can be obtained even in harsh environments such as dark nighttime or backlit conditions. Other issues, configurations, and advantages will become clearer through the following description of the embodiments.

[0012] FIG. 1 is a diagram showing the configuration of a video system according to an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of a camera unit according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a synthesized video according to an embodiment of the present invention. FIG. 4 is a diagram showing the configuration of a video display terminal and learning data collection device according to an embodiment of the present invention. FIG. 5 is a flowchart of processing executed by a video judgment unit according to an embodiment of the present invention. FIG. 6 is a diagram showing the relationship between the sun depression angle, field of view angle, and judgment angle used in an embodiment of the present invention. FIG. 7 is a diagram showing a judgment table by a video judgment unit according to an embodiment of the present invention. FIG. 8 is a diagram showing a judgment table by a video judgment unit according to an embodiment of the present invention. FIG. 9 is a flowchart of processing executed by a video judgment unit according to an embodiment of the present invention.

[0013] FIG. 1 is a diagram showing the configuration of a video system according to an embodiment of the present invention.

[0014] The video system of this embodiment is composed of a camera unit 101 and a video display terminal / learning data collection device 102. The video display terminal / learning data collection device 102 is equipped with a VMS (Video Management System) and monitors the video captured by the camera unit 101.

[0015] FIG. 2 is a diagram showing the configuration of the camera unit 101.

[0016] The camera unit 101 is composed of an imaging unit 201, a program memory 204, a flash memory 205, a CPU 206, a data bus 207, an internal clock 214, and a LAN interface 215. The imaging unit 201, the program memory 204, the flash memory 205, the CPU 206, the data bus 207, the internal clock 214, and the LAN interface 215 are connected by the data bus 207 so as to be able to communicate with each other.

[0017] The imaging unit 201 is an optical-electrical conversion device formed, for example, by a CMOS sensor, and captures an image by converting an optical signal transmitted through a lens (not shown) into an electrical signal. The flash memory 205 is a non-volatile semiconductor storage device that stores image data captured by the imaging unit 201 and data necessary for the operation of the camera unit 101. The CPU 206 is a computing device that executes programs stored in the program memory 204, and realizes the functions of the camera unit 101 by executing the programs.

[0018] The internal clock 214 synchronizes with the NTP server and corrects the time as needed (for example, every hour). The LAN interface 215 controls communication with other devices, acquires weather and precipitation data, and outputs the video data compressed by the video compression unit 212 to the video display terminal and learning data collection device 102.

[0019] The program memory 204 includes a lens control unit 202, a photometry area control unit 203, and a signal processing unit 208. More specifically, the program memory 204 stores programs that function as the lens control unit 202, the photometry area control unit 203, and the signal processing unit 208. The signal processing unit 208 includes an A / D conversion unit 209, a sharpening processing unit 210, an image synthesis unit 211, an image compression unit 212, and an image determination unit 213-1. The photometry area control unit 203 stores the photometry area setting set by the image determination unit 213-1. The lens control unit 202 controls the lens aperture using the data set in the photometry area control unit 203. The sharpening processing unit 210 performs sharpening processing according to the sharpening strength set by the image determination unit 213-1. The sharpening processing unit 210 may be configured using a machine learning model trained using pre-sharpening images and post-sharpening images of correct answer data 422 and incorrect answer data 423 collected by the image display terminal and learning data collection device 102 (described later). If the machine learning model of the sharpening processing unit 210 has been trained using the pre-sharpening image and the sharpening intensity, the sharpening processing is performed according to the sharpening intensity output by the machine learning model, and the sharpening processing performed by the sharpening processing unit 210 is, for example, a process of adjusting the brightness of the image, a process of emphasizing the edges of objects in the image, etc.

[0020] The camera unit 101 of the video system of this embodiment is characterized by an image synthesis unit 211 and an image determination unit 213-1. The image synthesis unit 211 synthesizes the image (before sharpening) output from the imaging unit 201 with the image (after sharpening) obtained by sharpening the image output from the imaging unit 201 by the sharpening processing unit 210, thereby generating a composite image 301. The image compression unit 212 compresses the composite image 301 and outputs the compressed image data from the LAN interface 215 to the video display terminal and learning data collection device 102.

[0021] 3 is a diagram showing an example of a composite image 301. The composite image 301 is an image obtained by arranging and combining the image (before sharpening) output from the imaging unit 201 and the image (after sharpening) output from the sharpening processing unit 210. The image (before sharpening) output from the imaging unit 201 and the image (after sharpening) output from the sharpening processing unit 210 may be displayed side by side, or may be displayed side by side, one above the other.

[0022] 5 and 6 are flowcharts of the process executed by the video determination unit 213-1.

[0023] The image determination unit 213-1 determines the image sharpness level based on the camera's installation location (latitude, longitude), camera orientation, weather, precipitation, sun position, and moon phase. The camera's installation location (latitude, longitude) and camera orientation are set when the camera is installed. Weather and precipitation information are obtained via the Internet from an external system (e.g., a weather information provider) or from a weather sensor installed near the camera unit 101.

[0024] The position of the sun can be calculated using the following formulas: Solar altitude α = arcsin {sin(latitude) sin(δ) + cos(latitude) cos(δ) cos(h)} Solar azimuth ψ = arctan [cos(latitude) cos(δ) sin(h) ÷ {sin(latitude) sin(α) - sin(δ)}] Solar declination δ = 0.006918 - 0.399912 cos(θo) + 0.070257 sin(θo) - 0.006758 cos(2θo) + 0.000907 sin(2θo) - 0.002697 cos(3θo) + 0.001480 sin(3θo) θo = 2π (number of days since New Year's Day - 1) ÷ 365 Solar hour angle h = (Japan Standard Time - 12) π ÷ 12 + longitude difference from the standard meridian + equation of time (Eq) Equation of time Eq = 0.000075 + 0.001868 cos(θo) - 0.032077 sin(θo) - 0.014615 cos(2θo) - 0.040849 sin(2θo)

[0025] The age of the moon can be calculated from the time elapsed from the new moon to the current time by storing the date of the new moon in advance in flash memory 205, reading the date and time of the most recent new moon from flash memory 205, and using the current time obtainable from internal clock 214. For example, in the case of September 2023, the date and time of the most recent new moon is 10:35 on September 15, and the lunar age at 19:35 on September 29 is 14 days and 9 hours, so 14 + 9 ÷ 24 = 14.375.

[0026] First, the video determination unit 213-1 sets the sharpening strength, which is a sharpening parameter, to standard, and executes a photometry area initialization procedure that sets the photometry area to the entire screen (501). Standard sharpening processing means that no sharpening processing is executed.

[0027] Next, the video determination unit 213-1 executes a day / night determination procedure to determine whether it is daytime or nighttime (502). For example, the internal clock 214 may determine whether it is daytime or nighttime based on whether it is within a preset time period (for example, determining that the period from 6:00 to 18:00 is daytime). Alternatively, the times of sunrise and sunset may be obtained from an external system via the Internet, and the period between sunrise and sunset may be determined as daytime, and the rest of the time as nighttime. Obtaining the times of sunrise and sunset allows for more accurate determination, but from the perspective of determining the level of sharpening, there is not much difference in the sharpened video between the two determination methods.

[0028] If the video determination unit 213-1 determines that it is nighttime in the daytime / nighttime determination procedure 502, it executes a rain determination procedure in which it determines whether it is raining by referring to the weather data (503).

[0029] If the image determination unit 213-1 determines that it is raining in the rain determination procedure 503, it executes a precipitation determination procedure to determine whether the precipitation is 20 mm or more (506). If the precipitation is 20 mm or more, it is likely to be a downpour, which will have a significant impact, such as making the image whitish. In this regard, since the impact on the image changes depending on the camera zoom (angle of view and distance to the subject), it is advisable to make the precipitation threshold setting changeable.

[0030] If the image determination unit 213-1 determines in the precipitation determination procedure 506 that the amount of precipitation is 20 mm or more, it executes a strong sharpening parameter setting procedure for setting the strength of sharpening to strong (509).

[0031] If the image determination unit 213-1 determines in the precipitation determination procedure 506 that the amount of precipitation is less than 20 mm, it executes a medium sharpening parameter setting procedure that sets the sharpening strength to medium (508).

[0032] If the image determination unit 213-1 determines that it is not raining in the rain determination procedure 503, it executes a fog determination procedure in which it determines whether it is foggy by referring to the weather in the weather data (504). In the fog determination procedure 504, it may determine whether it is foggy by using the weather data, or it may determine whether it is foggy by referring to the visibility distance observed by a weather sensor installed near the camera unit 101. In this case, for example, if the visibility distance is 50 m or more and less than 500 m, the sharpening strength may be set to medium; if the visibility distance is less than 50 m, the sharpening strength may be set to strong; and if the visibility distance is 500 m or more, the sharpening strength may remain standard.

[0033] If the image determination unit 213-1 determines that there is fog in the fog determination step 504, it executes a strong sharpening parameter setting step to set the sharpening strength to strong (509).

[0034] If the image determination unit 213-1 determines that the weather is not foggy in the fog determination procedure 504, it executes a clear weather determination procedure to determine whether the weather is clear (505). If the weather is not clear, that is, if the weather is determined to be cloudy, the sharpening strength remains standard.

[0035] If the video determination unit 213-1 determines that it is clear in the clear weather determination procedure 505, it executes a moon age determination procedure to compare the age of the moon with a predetermined threshold value and determine the brightness of the moon (507). The method of calculating the age of the moon is as described above.

[0036] If the image determination unit 213-1 determines in the moon age determination procedure 507 that the moon age is 6 or less or 23 or more, it executes a sharpening parameter medium setting procedure that sets the sharpening strength to medium (508). When the moon age is 6 or less or 23 or more, the moon is smaller than a half moon and becomes dark. However, since the effect of the moon's brightness changes depending on whether the surroundings are bright or dark, it is advisable to make the moon age threshold variable as needed.

[0037] If it is determined that it is nighttime in the daytime / nighttime determination step 502, the photometry area remains the entire area.

[0038] Next, referring to FIG. 6, a process when it is determined that it is daytime in the daytime / nighttime determination procedure 502 will be described.

[0039] As in step 503 when it is determined that it is nighttime, the video determination unit 213-1 executes a rain determination step of determining whether it is raining by referring to the weather data (513).

[0040] If the image determination unit 213-1 determines that it is raining in the rain determination procedure 513, it executes a precipitation determination procedure to determine whether the precipitation is 20 mm or more (516). If the precipitation is 20 mm or more, it is likely to be a downpour, which will have a significant impact, such as making the image whitish. In this regard, since the impact on the image changes depending on the camera zoom (angle of view and distance to the subject), it is advisable to make the precipitation threshold changeable through settings.

[0041] If the image determination unit 213-1 determines in the precipitation determination procedure 516 that the amount of precipitation is 20 mm or more, it executes a strong sharpening parameter setting procedure for setting the strength of sharpening to strong (519).

[0042] If the image determination unit 213-1 determines in the precipitation determination procedure 516 that the amount of precipitation is less than 20 mm, it executes a medium sharpening parameter setting procedure that sets the sharpening strength to medium (518).

[0043] If the video determination unit 213-1 determines that it is not rain in the weather: rain determination procedure 513, it executes a fog determination procedure in which it determines whether it is foggy by referring to the weather in the weather data (514). In the fog determination procedure 514, it may determine whether it is foggy by using the weather data, or it may determine whether it is foggy by referring to the visibility distance observed by a weather sensor installed near the camera unit 101. In this case, for example, if the visibility distance is 50 m or more and less than 500 m, the sharpening strength may be set to medium; if the visibility distance is less than 50 m, the sharpening strength may be set to strong; and if the visibility distance is 500 m or more, the sharpening strength may remain standard.

[0044] If the image determination unit 213-1 determines that there is fog in the fog determination step 514, it executes a strong sharpening parameter setting step to set the sharpening strength to strong (519).

[0045] If the image determination unit 213-1 determines that the weather is not foggy in the fog determination procedure 514, it executes a clear weather determination procedure to determine whether the weather is clear (515). If the weather is not clear, i.e., if the weather is cloudy, the sharpening strength remains standard.

[0046] If the video determination unit 213-1 determines that the weather is clear in the clear weather determination procedure 515, it executes a sun position determination procedure (520) to determine whether the sun's position is within a predetermined determination range. The determination range (determination angle) used for the determination can be set to the range obtained by adding the range in which the sun will enter the viewing angle of the camera unit 101 after a predetermined time has elapsed to the viewing angle. By setting a determination angle wider than the viewing angle, the lens can be narrowed before the sun enters the viewing angle, preventing saturation of the image level. This determination range is set when the camera unit 101 is installed. If the sun's position is outside the predetermined range, the sharpening strength remains standard. The relationship between the depression angle, viewing angle, and determination angle is as shown in FIG. 7.

[0047] If the image determination unit 213-1 determines that the position of the sun is within the field of view of the camera unit 101, it executes a sun altitude determination procedure (521) to compare the position of the sun with the optical axis of the camera unit 101. If the position of the sun is above the optical axis, the image determination unit 213-1 executes a strong sharpening parameter setting procedure (522) to set the sharpening strength to strong, and executes a lower metering area setting procedure (523) to set the metering area to the lower part of the field of view. By setting the metering area to the lower part of the field of view, the lens aperture is set to the brightness at a position within the field of view where the influence of sunlight is weak, preventing the image level from saturating.

[0048] On the other hand, if the sun is positioned below the optical axis, the image determination unit 213-1 executes a strong sharpening parameter setting procedure that sets the sharpening strength to strong (524), and executes an upper metering area setting procedure that sets the metering area to the upper part of the field of view (525). By setting the metering area to the upper part of the field of view, the lens aperture is set to the brightness at a position within the field of view where the effect of sunlight is weak, preventing the image level from saturating.

[0049] In the processing of the image determination unit 213-1 shown in FIGS. 5 and 6, day or night, weather, position of the sun, and age of the moon are used as parameters of the shooting environment, but some of the parameters may also be used.

[0050] The processing executed by the image determination unit 213-1 described above determines the sharpening strength and photometry area as shown in the determination tables of FIGS.

[0051] The process described above changes the learning model, which outputs different images depending on the sharpening level, and improves visibility by applying strong sharpening in environments with poor visibility and standard sharpening in environments with good visibility. Furthermore, by determining the photometry area based on the judgment range including the camera's field of view and the sun's position, the aperture can be adjusted before the sun enters the camera's field of view, improving image responsiveness and enabling the acquisition of images without blown-out highlights.

[0052] Next, the video display terminal and learning data collection device 102 will be described.

[0053] FIG. 4 is a diagram showing the configuration of the video display terminal and learning data collection device 102.

[0054] The video display terminal and learning data collection device 102 is composed of a program memory 404, a CPU 406, a data bus 407, an internal clock 414, a LAN interface 415, an auxiliary storage device 421, and a monitor output interface 427. The program memory 404, the CPU 406, the internal clock 414, the LAN interface 415, the auxiliary storage device (HDD) 421, and the monitor output interface 427 are connected by the data bus 407 so that they can communicate with each other.

[0055] The auxiliary storage device 421 is, for example, a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD). The auxiliary storage device 421 also stores data used by the CPU 406 when executing the program (e.g., correct answer data 422, incorrect answer data 423, and other logs) and the program executed by the CPU 406. That is, the program is read from the auxiliary storage device 421, loaded into the program memory 404, and executed by the CPU 406 to realize each function of the video display terminal and learning data collection device 102.

[0056] The program memory 404 stores programs that function as a display image cutout unit 424 , a learning image cutout unit 425 , a video determination unit 413 - 2 , and a video decoding unit 426 .

[0057] The CPU 406 is a computing device that executes the programs stored in the program memory 404, and realizes the functions of the video display terminal and learning data collection device 102 by executing the programs.

[0058] The internal clock 414 synchronizes with the NTP server and corrects the time as needed (for example, every hour). The LAN interface 415 is an interface that controls communication with other devices, acquires weather and precipitation data, and receives video data output from the camera unit 101. The monitor output interface 427 is an interface that outputs the results of calculations by the video display terminal and learning data collection device 102 as a visible sun.

[0059] The characteristic features of the video display terminal and learning data collection device 102 of this embodiment are that correct data 422 and incorrect data 423 are stored in an auxiliary memory device 421, and that it has a display image cutting unit 424, a learning image cutting unit 425, and a video judgment unit 413-2.

[0060] In this embodiment, the camera unit 101 outputs images before and after image sharpening. The display image cropping unit 424 crops the image after image sharpening on the right side of the composite image 301. The image decoding unit 426 decodes the image after image sharpening cropped from the right side of the composite image 301. The monitor output interface 427 outputs the decoded image data to enable monitoring of the image after sharpening. The learning image cropping unit 405 crops the image before image sharpening on the left side of the composite image 301, and generates data to be stored in the auxiliary storage device 421 as correct answer data 422 or incorrect answer data 423.

[0061] Similar to the video determination unit 213-1 in the camera unit 101, the video determination unit 213-2 determines the conditions of daytime, weather, precipitation, age of the moon, and solar position, classifies the data as correct answer data 422 or incorrect answer data 423, and stores the data as correct answer data 422 or incorrect answer data 423 in the auxiliary storage device 421. In addition, the correct answer data 422 and incorrect answer data 423 may be assigned a level of certainty of being correct or incorrect and stored in the auxiliary storage device 421.

[0062] The program executed by the CPU 406 is provided to the video display terminal and learning data collection device 102 from removable media (such as a CD-ROM or flash memory) or via a network, and is stored in a non-volatile auxiliary storage device 421, which is a non-transitory storage medium. For this reason, the video display terminal and learning data collection device 102 preferably has an interface for reading data from removable media.

[0063] The video display terminal and learning data collection device 102 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate on a virtual computer constructed on multiple physical computer resources. For example, each functional unit may operate on a separate physical or logical computer, or multiple functional units may be combined to operate on a single physical or logical computer.

[0064] 10 and 11 are flowcharts of the process executed by the video determination unit 413-2.

[0065] First, the video determination unit 413-2 executes a day / night determination procedure 802 to determine whether it is daytime or nighttime. For example, the internal clock 414 may determine whether it is daytime or nighttime based on whether it is within a preset time period (for example, determining that the period from 6:00 to 18:00 is daytime). Alternatively, the sunrise and sunset times may be obtained from an external system via the Internet, and the period between sunrise and sunset may be determined as daytime, and the remaining time as nighttime. Obtaining the sunrise and sunset times allows for more accurate determination. The internal clock 214 may be calibrated at regular intervals, so that there is no substantial time difference between the camera unit 101 and the video display terminal and learning data collection device 102.

[0066] If the video determination unit 413-2 determines that it is nighttime in the daytime / nighttime determination procedure 802, it executes a rain determination procedure in which it determines whether it is raining by referring to the weather data (803).

[0067] If it is determined that it is raining in the rain determination procedure 803, the image determination unit 413-2 executes a precipitation amount determination procedure to determine whether the amount of precipitation is 20 mm or more (806).

[0068] If the precipitation determination step 806 determines that the precipitation is 20 mm or more, the image determination unit 413-2 determines that the image classification is incorrect level 2, and saves the pre-image sharpening image cut out by the learning image cutout unit 405 as incorrect data 423 in an incorrect level 2 folder in the auxiliary storage device 421 (809). The incorrect data 423 is stored in folders by level in the auxiliary storage device 421, and incorrect level 2 is a label attached to learning images used under conditions where the strength of sharpening is strong.

[0069] If the image determination unit 413-2 determines that the precipitation amount is less than 20 mm in the precipitation amount determination procedure 806, it determines that the image classification is incorrect level 1, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in an incorrect level 1 folder in the auxiliary storage device 421 (808). Incorrect level 1 is a label attached to learning images used under medium sharpening conditions.

[0070] If the image determination unit 413-2 determines that it is not raining in the rain determination procedure 803, it executes a fog determination procedure in which it determines whether it is fog by referring to the weather in the weather data (804). In the fog determination procedure 804, it may determine whether it is fog by using the weather data, or it may determine whether it is fog by referring to the visibility distance observed by a weather sensor installed near the camera unit 101.

[0071] If the image determination unit 413-2 determines that the image is foggy in the fog determination procedure 804, it determines that the image classification is incorrect level 2, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in the incorrect level 2 folder in the auxiliary storage device 421 (809).

[0072] If the image determination unit 413-2 determines that it is not foggy in the fog determination procedure 804, it executes a clear weather determination procedure to determine whether the weather is clear (805). If the image determination unit 413-2 determines that the weather is not clear, i.e., that the weather is cloudy, it determines that the image classification is correct answer data night, and saves the image before image sharpening cut out by the learning image cutout unit 405 as correct answer data 422 in a night folder in the auxiliary storage device 421 (810).

[0073] If the image determination unit 413-2 determines that it is clear in the clear weather determination procedure 805, it executes a moon age determination procedure to compare the age of the moon with a predetermined threshold value and determine the brightness of the moon (807). The method of calculating the age of the moon is as described above.

[0074] If the image determination unit 413-2 determines in the lunar age determination step 807 that the lunar age is 6 or less or 23 or more, it saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in an incorrect answer level 1 folder in the auxiliary storage device 421 (808). When the lunar age is 6 or less or 23 or more, the moon is smaller than a half moon and is dark. However, since the effect of the brightness of the moon changes depending on whether the surroundings are bright or dark, it is advisable to make the threshold value of the lunar age changeable as necessary.

[0075] If the image determination unit 413-2 determines in the lunar age determination procedure 807 that the lunar age is greater than 6 and less than 23, it saves the image before image sharpening cut out by the learning image cutout unit 405 as correct answer data 422 in the night folder of the auxiliary storage device 421 (810).

[0076] Next, referring to FIG. 11, a process when it is determined that it is daytime in the daytime / nighttime determination procedure 802 will be described.

[0077] The video determination unit 413-2 executes a rain determination procedure (813) to determine whether it is raining by referring to the weather data, similar to the procedure 803 when it is determined that it is nighttime.

[0078] If it is determined that it is raining in the rain determination procedure 813, the image determination unit 413-2 executes a precipitation amount determination procedure to determine whether the amount of precipitation is 20 mm or more (816).

[0079] If the image determination unit 413-2 determines in the precipitation determination procedure 816 that the precipitation is 20 mm or more, it determines that the image classification is incorrect level 2, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in the incorrect level 2 folder in the auxiliary storage device 421 (819).

[0080] If the image determination unit 413-2 determines that the precipitation amount is less than 20 mm in the precipitation amount determination procedure 816, it determines that the image classification is incorrect level 1, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in the incorrect level 1 folder in the auxiliary storage device 421 (818).

[0081] If the image determination unit 413-2 determines that it is not raining in the rain determination procedure 813, it executes a fog determination procedure in which it determines whether it is fog by referring to the weather in the weather data (814). In the fog determination procedure 814, it may determine whether it is fog by using the weather data, or it may determine whether it is fog by referring to the visibility distance observed by a weather sensor installed near the camera unit 101.

[0082] If the image determination unit 413-2 determines that the image is foggy in the fog determination procedure 814, it determines that the image classification is incorrect level 2, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in the incorrect level 2 folder in the auxiliary storage device 421 (819).

[0083] If the image determination unit 413-2 determines that it is not foggy in the fog determination procedure 814, it executes a clear weather determination procedure to determine whether the weather is clear (815). If the image determination unit 413-2 determines that the weather is not clear, i.e., that the weather is cloudy, it determines that the image classification is correct answer data night, and saves the image before image sharpening cut out by the learning image cutout unit 405 as correct answer data 422 in a daytime folder in the auxiliary storage device 421 (822).

[0084] If the video determination unit 413-2 determines that the weather is clear in the clear weather determination procedure 815, it executes a sun position determination procedure to determine whether the position of the sun is within a predetermined determination range (820). The determination range (determination angle) used for the determination may be set to the range obtained by adding the viewing angle to the range within which the sun will fall within the viewing angle of the camera unit 101 after a predetermined time has elapsed. This determination range is set when the camera unit 101 is installed. If the position of the sun is outside the predetermined range, the video before image sharpening, which has been cut out by the learning image cutout unit 405, is saved as correct answer data 422 in a daytime folder in the auxiliary storage device 421 (820).

[0085] If the image judgment unit 413-2 determines that the position of the sun is within a predetermined judgment range, it judges that the image classification is incorrect level 2, and saves the image before image sharpening cut out by the learning image cutout unit 405 as incorrect data 423 in the incorrect level 2 folder of the auxiliary storage device 421 (819).

[0086] As shown in the figure, if it is determined that the position of the sun is within a predetermined determination range, a solar altitude determination procedure (821) may be executed to compare the position of the sun with the optical axis of the camera unit 101. However, since the type of image before sharpening cut out by the learning image cutout unit 405 does not change depending on the conclusion of the determination, the processing of procedure 821 does not need to be executed.

[0087] The method described above allows for classification of learning images by situation, thereby shortening the time required to collect and extract learning data. It is advisable to set an upper limit on the storage capacity of the learning video data to be collected, and when the amount of collected learning video data reaches this upper limit, delete one of the video data with overlapping evaluation values ​​representing the shooting environment, and use video data from a variety of shooting environments as learning data.

[0088] As described above, according to the embodiment of the present invention, the camera unit 101 automatically adjusts the sharpening parameters according to the shooting environment when shooting video, thereby improving the visibility of the captured image. In particular, in backlit situations, adjusting the sharpening parameters before the sun enters the field of view improves responsiveness and prevents the display of video with poor visibility.

[0089] Furthermore, the video display terminal and learning data collection device 102 labels image data that requires sharpening according to the shooting environment as incorrect data 423, and labels image data that does not require sharpening as correct data 422, so that appropriate labels can be automatically assigned to the learning data, thereby reducing the work time required to extract learning data for learning sharpening processing that suits the shooting environment.

[0090] In the processing of the image determination unit 413-2 shown in Figures 10 and 11, parameters of the shooting environment such as day or night, weather, position of the sun, and age of the moon are used, but some of the parameters may be used as long as they are the same as the parameters of the shooting environment used in the processing of the image determination unit 213-1 shown in Figures 5 and 6.

[0091] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0092] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0093] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC card, SD card, or DVD.

[0094] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected.

Claims

1. A data collection device for collecting image data, comprising a computer having an arithmetic unit that executes predetermined arithmetic processing and a storage device accessible by the arithmetic unit, wherein the arithmetic unit has an image judgment unit that outputs data in which image data is labeled according to parameters of the image shooting environment, and wherein the image judgment unit outputs data in which image data that requires sharpening is labeled as incorrect data and image data that does not require sharpening is labeled as correct data according to the shooting environment.

2. A data collection device according to claim 1, wherein the parameter of the photographing environment is at least one of the weather, the position of the sun, and the age of the moon.

3. A video system for collecting image data, comprising: a camera that captures video and outputs image data; and a data collection device that collects the image data captured by the camera, wherein the camera has a sharpening processing unit that sharpens the captured image data in accordance with the shooting environment of the image; and the data collection device has a video judgment unit that outputs data in which image data is labeled in accordance with parameters of the shooting environment of the image, and the video judgment unit labels image data that requires sharpening as incorrect data and outputs data in which image data that does not require sharpening is labeled as correct data in accordance with the shooting environment.

4. The imaging system according to claim 3, wherein the parameter of the imaging environment is at least one of weather, position of the sun, and age of the moon.

Images