Image recording device and image recording method

Abstract

To record a useful image while suppressing the use of memory capacity.SOLUTION: An image recorder (100) according to the present invention comprises: an image recognition part (120) which calculates recognition accuracy, that is, the accuracy of recognition of a recognition object from an image obtained by photographing the periphery of a vehicle, by using a recognition model (130) for recognizing a predetermined recognition object, and which determines that the recognition object is recognized when the recognition accuracy is a specified threshold or above; a recording control part (140) which records the taken image in a recording unit (150) when the recognition object is recognized; and a mode update part (160) that updates the recognition model (130) so as to increase the threshold of the recognition accuracy when the recognition object is recognized.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image recording apparatus and an image recording method. [Background technology]

[0002] When you pass an old car while driving, take a picture of the old car and save it as an image. Sometimes you want to record something. In that case, you need to take and record the image manually, which is time-consuming and requires safety measures. There was a problem. Here, Patent Document 1 discloses a technique related to a driving support device. The driving support device disclosed in Patent Document 1 detects the stop of a preceding vehicle from an external image captured while driving. It detects things like lights going out, and starts recording video when it detects the lights of the vehicle ahead are on. It is. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-221134 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, the image continues to be recorded every time image recognition is performed. There is a problem that the same vehicle is photographed and recorded multiple times, resulting in insufficient storage capacity. .

[0005] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide an image recording device and an image recording method for recording useful images while suppressing the use of memory capacity. [Means for solving the problem]

[0006] A first aspect of the present invention is an image recording device comprising an image recognition unit that uses a recognition model for recognizing a specified recognition target to calculate a recognition accuracy, which is the likelihood of recognizing the recognition target from a captured image of the vehicle's surroundings, and that determines that the recognition target has been recognized if the recognition accuracy is equal to or greater than a specified threshold, a recording control unit that records the captured image in a recording device when the recognition target is recognized, and a model update unit that updates the recognition model to increase the threshold of the recognition accuracy when the recognition target is recognized.

[0007] A second aspect of the present invention is an image recording method comprising an image recognition step of calculating a recognition accuracy, which is the likelihood of recognizing a predetermined recognition target from a captured image of the vehicle's surroundings, using a recognition model for recognizing the recognition target, and determining that the recognition target has been recognized if the recognition accuracy is equal to or greater than a predetermined threshold, a recording control step of recording the captured image in a recording device if the recognition target is recognized, and a model updating step of updating the recognition model to raise the recognition accuracy threshold if the recognition target is recognized. Effect of the Invention

[0008] According to the present invention, it is possible to provide an image recording device and an image recording method for recording useful images while suppressing the use of storage capacity. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a configuration of an image recording device according to a first embodiment of the present invention. [Diagram 2] 5 is a flowchart showing the flow of an image recording process according to the first embodiment. [Diagram 3] FIG. 11 is a block diagram showing the configuration of an image recording device according to a second embodiment of the present invention. [Figure 4] 10 is a flowchart showing the flow of an image recording process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. In the various drawings, the same elements are given the same reference numerals, and for clarity of explanation, Where appropriate, duplicate explanations will be omitted.

[0011] <Embodiment 1> FIG. 1 is a block diagram showing the configuration of an image recording device 100 according to the first embodiment. The image recording device 100 is, for example, a so-called drive recorder mounted on a moving object such as an automobile. The image recording device 100 includes an image acquisition unit 110, an image recognition unit 120, and a dictionary. The image forming apparatus includes a recording control unit 130, a recording control unit 140, a recording device 150, and a model update unit 160.

[0012] The image recording device 100 is connected to an on-board camera that captures images of the surroundings of the vehicle. The image acquisition unit 110 periodically acquires images captured by the vehicle-mounted camera and sends them to the image recognition unit 120. Output.

[0013] The dictionary 130 is an example of a recognition model for recognizing a predetermined recognition target, and is a trained model. Here, the recognition target is the object that you want to record and save as a collection. This is information that defines the two-dimensional shape features of an object. For example, popular car models, Old cars, famous places, historical sites, stores and other buildings, tourist spots and landscapes, people Examples of the recognition model include, but are not limited to, clothing. This can be achieved using a network or a support vector machine.

[0014] The dictionary 130 also stores a recognition result indicating whether or not the recognition target has been recognized from the input photographed image. In particular, when the recognition accuracy exceeds a threshold value, the dictionary 130 outputs the result. In other words, the dictionary 130 may determine that the captured image is recognized as a recognition target. If the target is in the image and can be recognized with a certainty above a certain value, it is selected as the target. It may be determined that the recognition was successful. In addition, the recognition result may include information about the recognition target contained in the captured image. The dictionary 130 may contain all the labels for two or more recognition objects. Alternatively, the dictionary 130 may be external to the image recording device 100 and may be stored in the image recording device. The recording device 100 may be connected via a communication line.

[0015] The image recognition unit 120 uses the dictionary 130 to recognize a recognition target from a captured image of the periphery of the vehicle. The image recognition unit 120 uses the dictionary 130 to recognize the features of the input image data. The system performs calculations using a set logic to calculate the likelihood (recognition accuracy) of the defined recognition target. The recognition accuracy is calculated as a value between 0 and 1, where 0 means the recognition is successful. 1 indicates that it is definitely a target, and the higher the number, the more likely it is. (High recognition accuracy). Recognition accuracy is expressed as a value from 0 to 100, for example. For example, the dictionary 130 receives a photographed image and calculates the number of images from the photographed image. The feature quantity is calculated from the calculated feature quantity, and a predetermined operation is performed on the feature quantity using the set parameters. The program modules and model formulas that implement the process of outputting the calculation results. The parameter is also called a weighting coefficient. The image recognition unit 120 records the recognition result. 40 and the model update unit 160.

[0016] When a recognition target is recognized in a captured image, the recording control unit 140 records the captured image in a recording device. The recognition result is recorded in the dictionary 130. This indicates that the recognition target is determined to be included in the captured image in The case where the character is recognized may be the case where the recognition accuracy exceeds a threshold value.

[0017] The recording device 150 is a non-volatile storage device such as a hard disk or a flash memory. The recording device 150 is a storage medium such as an SD card (registered trademark) or a slot into which the storage medium is inserted. The recording device 150 may be a controller that reads and writes the slot and storage medium. The photographed image 151 is stored. The photographed image 151 is associated with a photographing time or a recording time, etc. In addition, the photographed image 151 is labeled with the recognized object (recognition target). This is also fine.

[0018] If the recognition target is recognized and the recognition result satisfies a predetermined condition, the model update unit 160 After the captured image is recorded in the recording device 150, the recognition mode is set to suppress recognition of the recognition target. This updates the dictionary 130 so that if an equivalent recognition target is photographed in the future, In this case, recognition is suppressed, and therefore additional recording of the captured image 151 in the recording device 150 is suppressed. In other words, the recognition target is recognized, and the captured image is recorded in the recording device 150. In this case, subsequent recording of the same recognition target in the recording device 150 is suppressed. It is possible to record useful images while suppressing the use of capacity.

[0019] Here, the predetermined condition is that the recognition target is photographed with a size equal to or larger than a predetermined value in the photographed image. or when the recognition target has been photographed continuously for a certain period of time or longer. For example, if the recognition target is captured in a photograph at a certain size or larger, the recognition target may be It is sufficient as a record of the movement of the object to be recognized. If a group of images (videos) is captured, it can be said that it is sufficient to record the object to be recognized. By setting any of these as a predefined condition, it is possible to obtain high-quality images while suppressing subsequent additional recording. The size of the group can be, for example, the number of pixels of the entire captured image. The image is shot with 10% or more of the pixels, or 5% or more of the pixels that the object is to be recognized. It means that the image is taken with a certain value, for example, 100,000 pixels or more. For example, the period of time for which the image is shot is 30 seconds or more. Note that the predetermined conditions are not limited to those mentioned above. It won't be done.

[0020] Furthermore, if the predetermined condition is met, the recognition target included in the photographed image is photographed with a size equal to or larger than a predetermined value. In this case, the recording control unit 140 stores the previously recorded captured image 151 having a size less than the predetermined value. In this way, useful images are kept and unnecessary images are deleted, and the recording device 1 50 memory capacity can be effectively utilized.

[0021] In addition, the model update unit 160 updates the model when the recognition target is recognized and the recognition result satisfies a predetermined condition. In this case, it is advisable to update the dictionary 130 so as to raise the threshold for future recognition accuracy. Therefore, the recognition accuracy threshold will be raised in the next image recognition, so that the same recognition target will be recognized more frequently in the future. If a photograph is taken, it will not be recognized with the same or lower recognition accuracy, and unnecessary images will be recorded. Recording is suppressed.

[0022] In addition, the model update unit 160 updates the model when the ratio of the recognition target in the captured image is equal to or greater than a predetermined value. In this case, the model update unit 160 performs the process of updating the model when the recognition target is recognized and In addition, if the recognition result meets a certain condition, the ratio of the subsequent recognition targets is increased. For example, if the target object is included in the captured image but overlaps with other objects, the dictionary 130 is updated. In addition, if the object to be recognized is large in the captured image, it becomes difficult to recognize it. If the image contains a large proportion of the The quality of the captured images151 is improved because the higher the ratio of recognized objects to the total number of objects, the higher the quality of the captured images151. At this time, the recording control unit 140 detects the number of previously recorded images that is less than the predetermined ratio. Image 151 may be deleted.

[0023] Alternatively, the model update unit 160 may update the model if the recognition target is recognized and the recognition result satisfies a predetermined condition. If so, the recognition model may be updated so that the predetermined recognition target is not recognized thereafter. For example, the model update unit 160 may delete the recognition target from the dictionary 130. In this case, the recognition target may be excluded from the recognition targets in the dictionary 130.

[0024] The dictionary 130 stores a list of objects to be recognized, including a plurality of attributes and priorities for a particular object. For example, if the object to be recognized is a vintage car, attributes such as color, front, rear, left and right, and whether or not it has been modified may be used. Each attribute is assigned a priority in recognition. In this case, the recognition result includes the type of the object to be recognized as well as the attributes contained in the captured image. The model update unit 160 then checks whether the recognition result includes the above-mentioned attributes. For example, if the recognition result includes the attributes of the front and right side of a classic car in blue, In this case, the model update unit 160 updates the dictionary 130 with the following attributes of the old car: The priority of the characteristics "green", "forward", and "right" is updated to be lowered. The recognition and recording of images of classic cars with the attributes "green", "front", and "right side" are suppressed. Therefore, from then on, the dictionary 130 will not recognize that the photographed image being input is a color other than "blue" of a certain vintage car, or ... If the vehicle is on the side or behind the vehicle, it is judged to have been recognized (exceeded the recognition accuracy). For example, the priority is set based on the recognition accuracy threshold. Therefore, for the attributes of the recognition target that are already recorded in the recording device 150, The attributes that are not recorded will be recorded less frequently, and unrecorded attributes will be recorded with priority.

[0025] The image recording device 100 includes a processor and a memory, which are not shown in the figure. The storage device 150 also includes the image recognition unit 120, the recording control unit 140, and A computer program in which the processing of the model update unit 160 is implemented is stored. The processor then reads the computer program from the storage device 150 into the memory. The processor then executes the computer program. It realizes the functions of the recognition unit 120, the record control unit 140 and the model update unit 160.

[0026] FIG. 2 is a flowchart showing the flow of the image recording process according to the first embodiment. The image acquisition unit 110 acquires a captured image (S101). Next, the image recognition unit 120 Using the recognition model (dictionary 130), the recognition target is recognized from the captured image of the surroundings of the vehicle (S Then, the image recording device 100 judges whether the image recognition is successful or not (S1 03). In other words, it is determined whether the recognition target has been recognized or not.

[0027] If the recognition target is recognized, the recording control unit 140 records the captured image in the recording device 150. If the recognition target is recognized, the model update unit 160 updates the recognition result as follows (S104). It is determined whether or not a predetermined condition is satisfied (S105). If the recognition result satisfies the predetermined condition, The model update unit 160 updates the recognition model so as to suppress recognition of the recognition target (S106 ).

[0028] After that, the process returns to step S101. If the determination in step S103 or S105 is NO, If so, the process returns to step S101.

[0029] In this way, according to the present embodiment, it is possible to collect information about objects that a user wants to collect while driving a car or the like. When the vehicle passes through the designated area, image recognition is performed on the image captured by the vehicle-mounted camera using the dictionary 130. This allows images of the object to be automatically recorded and collected; and If a recorded image meets certain conditions, the recognition of similar images will be suppressed. In order to prevent the additional recording of the captured image 151 in the recording device 150, It is possible to record useful images while suppressing the use of memory capacity.

[0030] <Embodiment 2> The second embodiment is an improvement of the first embodiment. 1 is a block diagram showing a configuration of such an image recording device 100a. In comparison with the image recording device 100 of FIG. 1, the model update unit 160 is replaced with a model update unit 160a. Instead, an acquisition unit 170 has been added. The rest of the configuration is the same as in FIG. , duplicate explanations will be omitted as appropriate.

[0031] The acquisition unit 170 acquires new learning data for a recognition target from the outside. The recognition target may be, for example, an image of a new target object desired by the user, or an image of an object that has already been learned. Alternatively, the new recognition object may be an image of an additional attribute of the recognition object. The image may be an image of a stolen vehicle or a violator provided by the police or a local government. 0 acquires a group of images received from a source via a network, etc. as learning data. The acquisition unit 170 acquires a recognition model (dictionary 13) which is a trained model of a new recognition target from the outside. 0) may be obtained.

[0032] In this embodiment, the model update unit 160a updates the learning data acquired by the acquisition unit 170. The system includes a learning unit for performing learning using the data and updating the recognition model (dictionary 130). This allows the dictionary 130 to be effectively trained. The recognition model (dictionary 130) acquired by the acquisition unit 170 is used as a conventional recognition model. A modified embodiment may be one in which the above-mentioned is introduced in place of the Dell.

[0033] FIG. 4 is a flowchart showing the flow of image recording processing according to the second embodiment. The acquisition unit 170 acquires learning data distributed from an external source (S201). The dictionary update unit 160a learns using the learning data and updates the recognition model (dictionary 130). (S202).

[0034] In this way, in the second embodiment, new recognition targets provided from the outside are added to the dictionary 130. Incremental learning can make the dictionary 130 more effective. When conducting a search in a specific area, images of the objects to be searched are sent to a large number of vehicles in the specific area. By distributing the information simultaneously, the dictionary 130 of each vehicle can be instantly learned. This allows searches to be carried out more quickly and efficiently.

[0035] <Other embodiments> The image capturing apparatus further includes a position information acquiring unit (not shown) for acquiring position information at which the captured image was captured, The model update unit 160 updates the position information of the position where the recognition target is recognized by updating the position information of the position where the same recognition target was recognized in the past. The predetermined condition may be a match with the location information of the recognized location. This prevents background images from being recorded in duplicate, thus reducing memory usage. It is possible to record a wider variety of images while still capturing location information. For example, GNSS (Global Navigation System) This can be obtained through the Digital Ignition Satellite System.

[0036] The camera further includes a weather information acquisition unit (not shown) for acquiring weather information at the time the photographed image was taken. The model update unit 160 updates the weather information when the recognition target is recognized based on the weather information of the same recognition target in the past. The predetermined condition may be that the weather information is better than the weather information at the time of recognition. Even if the weather is bad, for example, on a rainy day when the image of the recognition object is not clear, the minimum level of accuracy is achieved. Recording is possible. And, even for the same recognition target, if the weather is equally bad, additional records will be recorded. Additional recording is possible only if the photo was taken in better weather conditions. For example, it can be obtained by communication from a server on the Internet.

[0037] In addition, when the image recognition unit 120 recognizes a recognition target from a captured image of the periphery of the vehicle, It is preferable that a notification unit (not shown) notifies the user that the recognition target has been recognized. Alternatively, the notification unit equipped with a monitor notifies the user by voice or image that the recognition target has been recognized. The notification unit notifies the user when the recording control unit 140 records the captured image in the recording device 150. The camera may be configured to notify the user that the captured image has been recorded.

[0038] The present invention has been described above based on the above embodiment. The present invention is not limited to the above, but is within the scope of the invention as defined in the claims of the present application. Of course, this includes various modifications, alterations, and combinations that can be made.

[0039] For example, in the above embodiment, the recognition target is recognized and the recognition result satisfies a predetermined condition. In this case, the model update unit updates the recognition model so that the same recognition target will not be recognized thereafter. However, if the recognition target is recognized and the recognition result satisfies a certain condition, The recording control unit suppresses recording, so that the same recognition target is not recorded thereafter. In this case, if the captured image contains a recognition target, it will be recognized but not recorded. As a result, the same effects as those of the above embodiment can be obtained.

[0040] Any of the above-mentioned processes of the communication device can be realized by making a CPU (Central Processing Unit) execute a computer program. In this case, the computer program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0041] In addition, a computer executes a program to realize the functions of the above-described embodiments. This not only realizes the functions of the above-mentioned embodiment, but also allows the program to be executed by a computer. The OS (Operating System) or application software running on the computer The embodiment of the present invention also includes a case where the functions of the above-mentioned embodiment are realized in cooperation with the above. Furthermore, all or part of the processing of this program may be implemented as an extension to a computer. The above-mentioned implementation is carried out by a function expansion unit connected to an expansion board or computer. Cases where the function of the form is realized are also included in the embodiments of the present invention. [Explanation of symbols]

[0042] 100 Image recording device 110 Image acquisition unit 120 Image Recognition Unit 130 Dictionaries 140 Recording control section 150 Recording Device 151 Images 160 Model Update Section

Claims

1. An image recognition unit that calculates a recognition accuracy, which is a likelihood of recognizing a predetermined recognition target from an image of the surroundings of a vehicle captured by a camera mounted on the vehicle, and is equipped with a recognition model that determines that the recognition target has been recognized when the recognition accuracy is equal to or greater than a predetermined threshold; a recording control unit that records the captured image in a recording device when the recognition target is recognized; a position information acquisition unit that acquires position information at the time when the photographed image was photographed by using a GNSS; a model update unit that updates the recognition model so as to increase the threshold value of the recognition accuracy when the recognition target is recognized from the captured image, the recognition result satisfies a predetermined condition, and the position information when the recognition target is recognized matches position information when the same recognition target was recognized in the past; An image recording device comprising:

2. the image recognition unit outputs a recognition result including attributes of the object to be recognized; the model update unit extracts the attribute included in the recognition result when the recognition target is recognized, and updates the recognition model for the recognition target including the extracted attribute so as to increase the threshold of the recognition accuracy.

2. The image recording device according to claim 1.

3. An image recognition step of calculating a recognition accuracy, which is the likelihood of recognizing a predetermined recognition target from an image of the surroundings of the vehicle captured by a camera mounted on the vehicle, and providing a recognition model which determines that the recognition target has been recognized if the recognition accuracy is equal to or greater than a predetermined threshold; a recording control step of recording the captured image in a recording device when the recognition target is recognized; a position information acquisition step of acquiring position information at the time when the photographed image was photographed by using a GNSS; a model updating step of updating the recognition model so as to increase the threshold value of the recognition accuracy when the recognition target is recognized from the captured image, the recognition result satisfies a predetermined condition, and the position information when the recognition target is recognized matches position information when the same recognition target was recognized in the past; An image recording method comprising:

Images