Tampering detection method, hash value calculation method, in-vehicle apparatus, hash value calculation program, and hash value storage system

Abstract

To provide a tampering detection method capable of reducing the processing load for calculating a hash value used to detect image tampering.SOLUTION: An exemplary tampering detection method for detecting tampering with a detection target image from an original image, comprises: applying the original image to a first AI model to calculate original image features; applying the original image features to a hash function to calculate a comparison reference hash value; applying the detection target image to a second AI model having the same processing content as the first AI model to calculate a detection target image features; applying the detection target image features to a detection hash function having the same processing content as the hash function to calculate a detection hash value; and determining tampering by comparing the comparison reference hash value with the detection hash value.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a tampering detection method, a hash value calculation method, an in-vehicle device, a hash value calculation program, and a hash value storage system. [Background technology]

[0002] In recent years, with the improvement in the performance of dashcams, footage from dashcams has been recognized as admissible evidence in trials involving traffic accidents, etc. Generally, dashcam footage is digital data that can be tampered with, so in court, the footage will only be recognized as evidence if it can be proven that the submitted video data has not been tampered with.

[0003] To prove that video data has not been tampered with, there is a digital forensic service that stores the hash value of the original video data (see, for example, Patent Document 1) in a secure area such as a server, compares the stored hash value with the hash value of the video data being investigated, and proves that the data has not been tampered with if the hash values ​​match. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-135418 A (paragraph 0061) Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the calculation cost for hash value calculation of large volumes of data such as video data is high, there is a risk that the resources of the computing device will be strained.

[0006] In view of the above circumstances, an object of the present invention is to provide a technique that can reduce the processing load for calculating a hash value used to detect image tampering. [Means for solving the problem]

[0007] An exemplary tampering detection method of the present invention is a tampering detection method that detects tampering of an original image with respect to a detection target image, and applies the original image to a first AI model to calculate original image features, applies the original image features to a hash function to calculate a comparison reference hash value, applies the detection target image to a second AI model with the same processing content as the first AI model to calculate detection target image features, applies the detection target image features to a detection hash function with the same processing content as the hash function to calculate a detection hash value, and determines whether tampering has occurred by comparing the comparison reference hash value with the detection hash value. [Effects of the Invention]

[0008] According to the exemplary embodiment of the present invention, the feature amount used as input to the hash function is smaller than the data size of the image, so the processing load for calculating the hash value used to detect image tampering can be reduced. [Brief explanation of the drawings]

[0009] [Figure 1] Diagram showing the configuration of the hash value storage system [Figure 2] FIG. 1 is a diagram showing a first embodiment of a drive recorder; [Figure 3] Diagram showing an overview of feature calculation processing [Figure 4] 1 is a flowchart showing an outline of the operation of a drive recorder; [Figure 5] FIG. 1 shows a first embodiment of a server. [Figure 6] Flowchart showing an outline of the hash value storage operation of the server [Figure 7] Flowchart showing an outline of the tamper detection operation of a server [Figure 8] FIG. 10 is a diagram showing a second embodiment of a drive recorder. [Figure 9] FIG. 2 illustrates a second embodiment of the server. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the contents of the embodiments shown below.

[0011] <1. Configuration of hash value storage system> FIG. 1 is a diagram showing the configuration of a hash value storage system SYS1 according to an exemplary embodiment of the present invention.

[0012] The hash value storage system SYS1 includes a server 100 and a drive recorder 1. In this embodiment, the server 100 is installed outside the vehicle V1, for example, in a hash value storage service center building, and the drive recorder 1 is an on-board device installed in the vehicle V1.

[0013] The server 100 may be a physical server or a virtual server. The server 100 may be configured by one server or by multiple servers. The server 100 and the drive recorder 1 communicate with each other via a network N1.

[0014] The drive recorder 1 is disposed at an appropriate location on the vehicle V1. The drive recorder 1 acquires images of the vehicle's surroundings captured by a camera 2 mounted on the vehicle V1. In the exemplary embodiment of the present invention, the drive recorder 1 includes the camera 2. In addition, in the exemplary embodiment of the present invention, the drive recorder 1 employs a continuous recording method in which information relating to the driving situation, including images captured by the camera 2, is periodically recorded on an external storage medium (not shown in FIG. 1 ).

[0015] In an exemplary embodiment of the present invention, the drive recorder 1 is driven by power supplied from a battery (not shown) included in the vehicle V1. In addition, in an exemplary embodiment of the present invention, the drive recorder 1 starts driving in response to an ignition (IG) being turned on. In addition, the drive recorder 1 stops driving in response to an IG being turned off. Note that this is an example, and the drive recorder 1 may switch between starting and stopping driving in conjunction with, for example, the on / off of an ACC (accessory).

[0016] 2. First embodiment of drive recorder Fig. 2 is a diagram showing a first embodiment of the drive recorder 1. The drive recorder 1 shown in Fig. 2 includes a camera 2, a microphone 3, an operation unit 4, a speaker 5, a controller 6, a memory 7, a card slot 8, a timing circuit 9, an acceleration sensor 10, a GPS (Global Positioning System) receiving unit 11, and a communication unit 12.

[0017] The camera 2 includes a lens and an imaging element, and electronically acquires image data. The camera 2 outputs the acquired image data to the controller 6. In this embodiment, the camera 2 is disposed near the top of the windshield with its optical axis directed forward of the vehicle V1. The number and arrangement of the cameras 2 may be changed as appropriate. For example, a front camera that captures images ahead and a rear camera that captures images behind may be provided.

[0018] The microphone 3 collects sounds around the vehicle and acquires audio data. The microphone 3 outputs the acquired audio data to the controller 6. The microphone 3 is disposed, for example, in the main body of the drive recorder 1 or in an appropriate position of the vehicle V1 away from the main body of the drive recorder 1.

[0019] The operation unit 4 receives instructions from the driver etc. The operation unit 4 is disposed in an appropriate position in the vehicle V1, such as near a steering wheel (not shown), so that the driver etc. can easily operate it.

[0020] The speaker 5 converts the audio signal output from the controller 6 into sound and outputs the sound. The speaker 5 is disposed, for example, in the main body of the drive recorder 1 or in an appropriate position in the vehicle V1 away from the main body of the drive recorder 1.

[0021] The controller 6 and the memory 7 are specifically computer devices.

[0022] The controller 6 controls the entire drive recorder 1. The controller 6 includes an arithmetic circuit that performs arithmetic processing. More specifically, the arithmetic circuit includes a processor. The processor includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. The controller 6 may be configured with one processor or multiple processors. When the controller 6 is configured with multiple processors, the processors are arranged to be able to communicate with each other.

[0023] The memory 7 is configured to include a volatile memory and a non-volatile memory. The volatile memory is specifically a RAM (Random Access Memory). The non-volatile memory is specifically a ROM (Read Only Memory). The non-volatile memory may also be a flash memory, a hard disk drive, or the like. The non-volatile memory stores computer-readable programs and data.

[0024] 2, the controller 6 includes, as its functions, an image acquisition unit 6a, an image processing unit 6b, and a warning processing unit 6c. The functions of the controller 6 are realized by a processor executing arithmetic processing in accordance with a program stored in a memory 7. The number of programs that realize the functions of the controller 6 may be one or more.

[0025] The program stored in memory 7 is a computer program that causes a computer to realize the functions of controller 6. Such a computer program may be provided, for example, by a computer-readable nonvolatile recording medium. The nonvolatile recording medium may be, for example, the nonvolatile memory described above, as well as an optical recording medium (e.g., an optical disk), a magneto-optical recording medium (e.g., a magneto-optical disk), a USB memory, or an SD card. As another example, the computer program may be provided from a program providing server via a communication line such as the Internet, or provided by so-called downloading.

[0026] Furthermore, each of the functional units 6a to 6c may be realized by a single program, or, for example, each functional unit may be realized by a separate program. As described above, each of the functional units 6a to 6c may be realized by a processor executing a program, i.e., by software, but may also be realized by other methods. Each of the functional units 6a to 6c may be realized using, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). That is, each of the functional units 6a to 6c may be realized by hardware using a dedicated IC or the like. Each of the functional units 6a to 6c may also be realized by a combination of software and hardware. Each of the functional units 6a to 6c is a conceptual component. The function performed by one component may be distributed among multiple components. Furthermore, the functions of multiple components may be integrated into one component.

[0027] The image acquisition unit 6a sequentially acquires images of each frame periodically transmitted from the camera 2.

[0028] The image processing unit 6b sequentially processes the images acquired by the image acquisition unit 6a. The image processing in the image processing unit 6b uses an artificial intelligence (AI) model M1 stored in the memory 7. In this embodiment, object detection (detection of vehicles, etc.) is performed by image processing using the AI ​​model M1.

[0029] In this embodiment, the AI ​​model M1 is a neural network. Specifically, the AI ​​model M1 is a convolutional neural network that has been trained using deep learning. The trained neural network is obtained, for example, by performing training using a training dataset, which is a collection of training data, using a known training method such as backpropagation. The memory 7 stores the structure and parameters of the trained neural network, as well as code instructions for executing the neural network, as the AI ​​model M1.

[0030] The warning processing unit 6c performs warning processing using the result of object detection (detection of a vehicle, etc.) performed by the image processing unit 6b. For example, the warning processing unit 6c estimates the distance between the vehicle detected by the image processing unit 6b and vehicle V1, and outputs an audio signal corresponding to a warning sound to the speaker 5 when the distance between the vehicles is less than a threshold.

[0031] The card slot 8 provides an interface between the controller 6 and an external storage medium. Specifically, the external storage medium is a removable memory card MC1. The card slot 8 is configured to allow the memory card MC1 to be attached and detached. The memory card MC1 is an example of an external storage medium. Note that the external storage medium is not limited to the memory card MC1. For example, the external storage medium may be a hard disk drive, an optical disk, a magneto-optical disk, a flexible disk, or the like. When an external storage medium other than the memory card MC1 is used, a component replacing the card slot 8 is provided in the drive recorder 1. For example, an optical disk device or the like is provided in the drive recorder 1 instead of the card slot 8.

[0032] By inserting the memory card MC1 into the card slot 8, it becomes possible to read data from the memory card MC1 and write data to the memory card MC1. In this embodiment, image data, audio data, etc. temporarily stored in the controller 6 are periodically recorded on the memory card MC1 inserted into the card slot 8. The image data temporarily stored in the controller 6 is provided with additional information such as vehicle information at the time the image data was captured by the camera 2, date and time, etc. The vehicle information includes, for example, the speed, acceleration, position, etc. of the vehicle V1.

[0033] The clock circuit 9 generates a signal corresponding to the current time and outputs it to the controller 6. The clock circuit 9 may have, for example, an internal battery, so that it can operate and keep accurate time without receiving an external power supply.

[0034] The acceleration sensor 10 detects the acceleration applied to the vehicle V1. Specifically, the acceleration sensor 10 detects the components of acceleration in three or two mutually orthogonal axes as instantaneous values, and acquires the magnitude and direction of the acceleration applied to the vehicle V1. The acceleration applied to the vehicle V1 includes not only gravitational acceleration but also acceleration due to an impact such as an accident. Therefore, the detection value of the acceleration sensor 10 can be used as an index of the magnitude of the impact applied to the vehicle V1. The acceleration sensor 10 outputs a signal corresponding to such acceleration to the controller 6.

[0035] The GPS receiver 11 receives signals from multiple GPS satellites to acquire the vehicle position, which is the position of the vehicle V1 at the current time. The GPS receiver 11 acquires the vehicle position as position information expressed in longitude and latitude on the Earth and outputs it to the controller 6. The GPS receiver 11 can detect the accurate time based on the signals from the GPS satellites. For this reason, the GPS receiver 11 may be used in place of the timing circuit 9. The GPS receiver 11 may also be used to correct the time of the timing circuit 9.

[0036] Additionally, vehicle information detected by a vehicle information sensor SEN1 is input to the controller 6 via an in-vehicle network NW1 such as a CAN bus. The vehicle information sensor SEN1 includes a vehicle speed sensor SEN2 that detects the speed of the vehicle V1, and a steering angle sensor SEN3 that detects the steering angle of the vehicle V1.

[0037] The communication unit 12 communicates with the server 100 via the network N1. The communication unit 12 transmits a feature map (specifically, a feature map after pooling) obtained after the convolution process in a convolution layer (k-th convolution layer) Lk (described later) to the server 100 via the network N1. Note that k is an arbitrary natural number between 1 and n. Here, k is preferably 2 or greater. If k is 2 or greater, the feature map (specifically, a feature map after pooling) transmitted to the server 100 will be a feature map obtained when the AI ​​model M1 performs the convolution process multiple times, and will be smaller in size than the feature map data obtained when the AI ​​model M1 performs the convolution process only once. This can facilitate a reduction in the processing load for calculating a hash value. Furthermore, the communication unit 12 may transmit to the server 100 via the network N1 a feature map (specifically, a feature map after pooling) obtained after the convolution process in a convolution layer (k-th convolution layer) Lk (described later) with the number of channels thinned out. This can facilitate a reduction in the processing load for calculating a hash value. In this case, the algorithm for how to thin out the number of channels can be understood by both the drive recorder 1 and the server 100. The method for thinning out the number of channels may change dynamically. If the method for thinning out the number of channels changes dynamically, even if a tamperer learns the AI ​​model M1 and the hash function, the tampering can be properly detected as long as the algorithm for how to thin out the number of channels is not known.

[0038] FIG. 3 is a diagram illustrating an overview of the feature calculation process executed by the image processing unit 6b. In FIG. 3, an input image 200 is an input image input to an AI model M1. As shown in FIG. 3, the AI ​​model M1 includes multiple convolution layers L1 to Ln. The variable n shown in FIG. 3 is 2 or greater. Although not shown in FIG. 3, a pooling layer is disposed after each of the convolution layers L1 to Ln. That is, the AI ​​model M1 forms deep layers by repeating a convolution layer and a pooling layer multiple times.

[0039] The convolutional layers L1 to Ln play a role in extracting local features of the input image 200. The first convolutional layer (first convolutional layer) L1 performs convolution processing on the input image 200 to obtain a feature map. The second and subsequent convolutional layers L2 to Ln perform convolution processing using the feature map obtained in the previous layer as input to obtain new feature maps. The feature map is tensor data output by the convolutional operation, and is data including the feature amounts extracted by the convolutional operation.

[0040] The pooling layer described above serves to consolidate and reduce the feature maps output from the convolution layer. The pooling process performed by the pooling layer may be a well-known method such as max pooling, which selects the maximum value, or average pooling, which selects the average value.

[0041] In addition, in FIG. 3, the dashed lines in each of the convolutional layers L1 to Ln indicate the results (bundles of feature maps) obtained by the convolutional processing. The symbols "Wm" and "Hm" indicate the size of the feature map. In detail, the symbol "Wm" indicates the width (horizontal size) of the feature map. The symbol "Hm" indicates the height (vertical size) of the feature map. The symbol "Cm" indicates the number of channels. In FIG. 3, the number of channels corresponds to the number of feature maps. In the example shown in FIG. 3, the size of the feature map is reduced and the number of channels is increased by repeating the convolutional processing by the convolutional layers.

[0042] The output layer LOUT executes processing using the feature quantities obtained through multiple convolution processes and outputs the image processing results. The output layer LOUT is configured using, for example, a fully connected layer. The fully connected layer receives as input the feature map (more specifically, the feature map after pooling) obtained after the convolution process in the final convolution layer (nth convolution layer) Ln. Since the feature map is two-dimensional information, it is expanded into one-dimensional information and input to the fully connected layer. The fully connected layer calculates weighted connections and determines the values ​​of units (neurons) using an activation function such as a rectified linear unit (ReLU). The output layer LOUT outputs, for example, whether or not a "vehicle" is present in the image as the image processing result.

[0043] The output layer LOUT does not necessarily have to be configured using a fully connected layer. Furthermore, the output layer LOUT may receive not only the feature values ​​(feature map) obtained after the final convolution process, but also the feature values ​​obtained by the convolution processes performed before that. To realize such a configuration, a known object detection algorithm such as YOLO may be used.

[0044] For example, when YOLO is used, the position (area) of a pre-specified detection target (object) and a reliability score indicating the probability of the presence of the detection target are output as image processing results. The reliability score is, for example, a numerical value between 0 and 1, and the closer the value is to "1," the higher the reliability is. In this embodiment, since the target object is a "vehicle," when the image processing is object detection, the position of the vehicle and a reliability score indicating the probability of the presence of the vehicle are output as image processing results.

[0045] Next, a specific example of the operation of the drive recorder 1 will be described. Fig. 4 is a flowchart showing the general operation of the drive recorder 1. The flow shown in Fig. 4 is started when the drive recorder 1 starts driving in response to IG ON.

[0046] In step S1, the controller 6 (image acquisition unit 6a) determines whether or not it has acquired a frame image from the camera 2. The camera 2 periodically transmits frame images to the controller 6. The controller 6 (image acquisition unit 6a) sequentially acquires each frame image transmitted from the camera 2. If a frame image has been acquired (Yes in step S1), the process proceeds to the next step S2. If a frame image has not been acquired (No in step S1), the process continues with step S1. If a frame image has not been acquired for a long time, an error notification may be sent to the user since there may be a malfunction or other problem.

[0047] In step S2, the controller 6 (image processing unit 6b) generates a compressed image by lossy compressing the frame image (original image) transmitted from the camera 2. Once the compressed image is generated, the process proceeds to the next step S3.

[0048] In step S3, the controller 6 (image processing unit 6b) generates a decoded image by decoded the compressed image generated in step S2. Once the decoded image is generated, the process proceeds to the next step S4. By executing the processes of steps S2 and S3, even if the image to be detected in the alteration detection is a decoded image obtained by decoded a compressed image compressed using lossy compression, alteration detection becomes possible by the comparison process of step S204, which will be described later. Note that if the image to be detected in the alteration detection is a decoded image obtained by decoded a compressed image compressed using lossless compression, or if the image is not a decoded image, the processes of steps S2 and S3 may be omitted.

[0049] In step S4, the controller 6 (image processing unit 6b) performs image processing on the decoded image using the AI ​​model M1. The image processing outputs an inference result related to a preset target object (specific object). In this embodiment, the target object is a "vehicle." The image processing outputs an inference result (image processing result) as to whether or not a vehicle is captured in the image. Once the image processing is performed, the process proceeds to the next step S5.

[0050] In step S5, the controller 6 (warning processing unit 6c) performs warning processing according to the image processing result. For example, if the image processing result is "a vehicle is present in the image," the controller 6 (warning processing unit 6c) estimates the distance between the vehicle in the image and vehicle V1, and outputs an audio signal corresponding to a warning sound to the speaker 5 when the distance is less than a threshold. If no vehicle is present in the image, or if the distance between the vehicle in the image and vehicle V1 is equal to or greater than the threshold, the controller 6 (warning processing unit 6c) does not perform processing such as outputting an audio signal. Once processing according to the image processing result has been performed, the process proceeds to the next step S6.

[0051] In step S6, the communication unit 12 transmits the feature map (more specifically, the feature map after pooling) obtained after the convolution process in the above-described convolution layer (k-th convolution layer) Lk to the server 100 via the network N1. After the feature map has been transmitted, the process proceeds to the next step S7. Note that in the flow shown in FIG. 4, feature maps for one frame are transmitted sequentially, but feature maps for multiple frames may be transmitted together. When transmitting the feature map, the feature map is transmitted in a state in which the feature map is associated with identification information of the drive recorder 1, the vehicle V1, the user, etc., and information for identifying the image of the frame corresponding to the feature map (such as the shooting time).

[0052] In step S7, the controller 6 stores the compressed image generated in step S2 in the memory card MC1. After the storage process of the compressed image in the memory card MC1 is completed, the process proceeds to the next step S8. Note that in the flow shown in Fig. 4, the compressed image is written sequentially to the memory card MC1 frame by frame, but compressed images for multiple frames may be written together to the memory card MC1.

[0053] In step S8, the controller 6 determines whether or not a reason for terminating the flow shown in Fig. 4 has occurred (termination reason). Examples of the termination reason include a command to terminate operation from the user and the occurrence of a malfunction such as a breakdown. If a termination reason has occurred (Yes in step S8), the flow shown in Fig. 4 is terminated. If a termination reason has not occurred (No in step S8), the process returns to step S1, and the processing from step S1 onwards is carried out. Note that when returning from step S8 to step S1, the image to be processed is the image of the frame acquired next to the image of the previously processed frame.

[0054] 3. First embodiment of the server Fig. 5 is a diagram illustrating a first embodiment of the server 100. The server 100 illustrated in Fig. 5 constitutes a hash value storage system SYS1 together with the drive recorder 1 illustrated in Fig. 2. The server 100 illustrated in Fig. 5 includes a controller 101, a memory 102, and a communication unit 103.

[0055] The hash value storage system SYS1, which is configured by the server 100 shown in FIG. 5 and the drive recorder 1 shown in FIG. 2, not only stores hash values ​​but also detects tampering.

[0056] The controller 101 and the memory 102 are specifically computer devices.

[0057] The controller 101 controls the entire server 100. The controller 101 includes an arithmetic circuit that performs arithmetic processing. More specifically, the arithmetic circuit includes a processor. The processor includes, for example, a CPU, a GPU, and the like. The controller 101 may include one processor or multiple processors. When the controller 101 includes multiple processors, the processors are arranged to be able to communicate with each other.

[0058] The memory 102 is configured to include volatile memory and nonvolatile memory. The volatile memory is specifically RAM. The nonvolatile memory is specifically ROM. The nonvolatile memory may also be a flash memory, a hard disk drive, or the like. The nonvolatile memory stores programs and data that can be read by a computer.

[0059] The communication unit 103 communicates with the drive recorder 1 via the network N1. The communication unit 103 receives, via the network N1, a feature map (more specifically, a feature map after pooling) obtained after the convolution process in the above-described convolution layer (k-th convolution layer) Lk.

[0060] 5, the controller 101 includes, as its functions, a hash value calculation unit 101a, an image acquisition unit 101b, an image processing unit 101c, a comparison unit 101d, and a determination unit 101e. The functions of the controller 101 are realized by a processor executing arithmetic processing in accordance with a program stored in a memory 102. The number of programs that realize the functions of the controller 101 may be one or more.

[0061] The program stored in memory 102 is a computer program that causes a computer to realize the functions of controller 101. Such a computer program may be provided, for example, by a computer-readable nonvolatile recording medium. The nonvolatile recording medium may be, for example, the nonvolatile memory described above, as well as an optical recording medium (for example, an optical disk), a magneto-optical recording medium (for example, a magneto-optical disk), a USB memory, or an SD card. As another example, the computer program may be provided from a program providing server via a communication line such as the Internet, or provided by so-called downloading.

[0062] Furthermore, each of the functional units 101a to 101e may be realized by a single program, or, for example, each functional unit may be realized by a separate program. As described above, each of the functional units 101a to 101e may be realized by having a processor execute a program, i.e., by software, but may also be realized by other methods. Each of the functional units 101a to 101e may be realized using, for example, an ASIC, an FPGA, or the like. That is, each of the functional units 101a to 101e may be realized by hardware using a dedicated IC, or the like. Each of the functional units 101a to 101e may also be realized by a combination of software and hardware. Each of the functional units 101a to 101e is a conceptual component. The function performed by one component may be distributed among multiple components. Furthermore, the functions of multiple components may be integrated into one component.

[0063] The hash value calculation unit 101a inputs the feature map (more specifically, the feature map after pooling) received by the communication unit 103 into a hash function to calculate a hash value. The hash function is a hash function identified by hash function information INF2 stored in the memory 102. The hash function information INF2 may be the hash function itself or identification information of the hash function. When the hash function information INF2 is identification information of the hash function, for example, the server 100 may use the hash function information INF2 to obtain a hash function corresponding to the hash function information INF2 from an external database.

[0064] The hash value calculation unit 101a inputs a feature map (specifically, a feature map after pooling) obtained from the image of the detection target into a hash function to calculate a hash value (a hash value for detection). The hash function is also a hash function specified by hash function information INF2 stored in the memory 102.

[0065] The image acquisition unit 101b sequentially acquires the image of each frame of the detection target. The image of each frame of the detection target is received by the communication unit 103, for example.

[0066] The image processing unit 101c sequentially processes images of the detection target acquired by the image acquisition unit 101b. The image processing unit 101c uses an AI model identified by AI model information INF1 stored in the memory 102 for image processing. The AI ​​model identified by the AI ​​model information INF1 has the same configuration as the AI ​​model M1. The AI ​​model information INF1 may be the AI ​​model itself or identification information for the AI ​​model. When the AI ​​model information INF1 is identification information for the AI ​​model, for example, the server 100 may use the AI ​​model information INF1 to acquire the AI ​​model corresponding to the AI ​​model information INF1 from an external database.

[0067] The image processing unit 101c may output an image processing result (object detection result) for the image of the detection target, but it is sufficient to at least generate a feature map (more specifically, a feature map after pooling) obtained after convolution processing in the convolution layer (k-th convolution layer) Lk for the image of the detection target.

[0068] The comparison unit 101d compares, on a frame-by-frame basis, a hash value calculated by inputting the feature map received by the communication unit 103 (specifically, the feature map after pooling) into a hash function, with a hash value (detection hash value) calculated by inputting the feature map obtained from the image to be detected (specifically, the feature map after pooling) into the hash function. Note that the comparison unit 101d may sample feature maps corresponding to images acquired at the same timing and compare the sampled feature maps on a frame-by-frame basis, rather than comparing all frames on a frame-by-frame basis.

[0069] If there is even one mismatch in the comparison results of the comparison unit 101d, the determination unit 101e determines that the image to be detected is a tampered image. Note that, taking into consideration the possibility of image data corruption due to noise or the like, if the mismatch rate in the comparison results of the comparison unit 101d is equal to or greater than a threshold value (for example, 1%), the determination unit 101e may determine that the image to be detected is a tampered image.

[0070] Next, a specific example of the operation of the server 100 will be described. Fig. 6 is a flowchart showing an outline of the hash value storage operation of the server 100. The flow shown in Fig. 6 starts when the power of the server 100 is turned on, for example.

[0071] In step S101, the controller 101 determines whether the communication unit 103 has received a feature map (specifically, the feature map after pooling). If the feature map (specifically, the feature map after pooling) has been received (Yes in step S101), the process proceeds to the next step S102. If the feature map (specifically, the feature map after pooling) has not been received (No in step S101), the process of step S101 continues. As described above, when the communication unit 12 included in the drive recorder 1 transmits the feature map, the feature map is transmitted in a state in which it is associated with identification information of the drive recorder 1, the vehicle V1, the user, etc., and information for identifying an image of a frame corresponding to the feature map (such as the shooting time). Therefore, when the communication unit 103 included in the server 100 receives the feature map, the feature map is received in a state in which it is associated with identification information of the drive recorder 1, the vehicle V1, the user, etc., and information for identifying an image of a frame corresponding to the feature map (such as the shooting time).

[0072] In step S102, the controller 101 (hash value calculation unit 101a) inputs the feature map (specifically, the feature map after pooling) received by the communication unit 103 into a hash function to calculate a hash value. The hash function is a hash function identified by hash function information INF2 stored in the memory 102. Because the size of the feature map is reduced by repeated convolution processing by the convolutional layer in the AI ​​model M1, inputting the feature map (specifically, the feature map after pooling) into the hash function to calculate a hash value reduces the processing load for calculating the hash value compared to inputting image data into the hash function to calculate a hash value. For example, when RGB 8-bit color image data having a width (horizontal size) of W pixels and a height (vertical size) of H pixels is input into the hash function, the input data size of the hash function is W × H × 3 × 8 [bits]. On the other hand, for example, if a feature map (specifically, a feature map after pooling) with a width (horizontal size) of W / 32 pixels, a height (vertical size) of H / 32 pixels, and 32 channels is input to the hash function, the input data size of the hash function will be W / 32 × W / 32 × 256 × 8 [bits]. By inputting a feature map (specifically, a feature map after pooling) with a height (vertical size) of H / 32 pixels and 32 channels to the hash function instead of an 8-bit RGB color image with a width (horizontal size) of W pixels and a height (vertical size) of H pixels, the calculation cost of the hash function can be reduced to 1 / 12 (= {W / 32 × W / 32 × 256 × 8} / {W × H × 3 × 8}). Once the hash value is calculated, processing proceeds to the next step, S103.

[0073] In step S103, the controller 101 stores the hash value HV calculated in step S102 in the memory 102. Once the hash value HV has been stored, the process proceeds to the next step, S104. The hash value HV is stored in the memory 102 in a state linked to identification information linked to the feature map used to calculate the hash value HV and information for identifying the image of the frame (such as the shooting time). In other words, the memory 102 also stores the identification information linked to the hash value HV and information for identifying the image of the frame (such as the shooting time). Note that, although hash values ​​for one frame are stored sequentially in the flow shown in FIG. 6, hash values ​​for multiple frames may be stored together.

[0074] In step S104, the controller 101 determines whether a reason for terminating the flow shown in Fig. 6 has occurred (a reason for termination). Examples of the reason for termination include a command to terminate operation from a user and the occurrence of a malfunction such as a breakdown. If a reason for termination has occurred (Yes in step S104), the flow shown in Fig. 6 is terminated. If a reason for termination has not occurred (No in step S104), the process returns to step S101, and the processes from step S101 onwards are performed. Note that when returning from step S104 to step S101, the feature map to be processed (more specifically, the feature map after pooling) is the feature map (more specifically, the feature map after pooling) received next to the previously received feature map (more specifically, the feature map after pooling).

[0075] Fig. 7 is a flowchart showing an outline of the tampering detection operation of the server 100. The flow shown in Fig. 7 is started when a user issues an instruction to start the tampering detection operation.

[0076] In step S201, it is determined whether the controller 101 (image acquisition unit 101b) has acquired an image of the detection target. The image of the detection target is a restored image obtained by restoring a compressed image stored in memory card MC1 in the controller 101 (image acquisition unit 101b). For example, a card slot may be provided in the server 100, and the controller 101 (image acquisition unit 101b) may read the compressed image from memory card MC1, or the compressed image read from memory card MC1 may be transmitted from an external device and received by the communication unit 103. If the image of the detection target has been acquired (Yes in step S201), the process proceeds to the next step S202. If the image of the detection target has not been acquired (No in step S201), the process of step S201 continues. In this embodiment, the image of the detection target is a moving image and is therefore composed of multiple frame images.

[0077] In step S202, the controller 101 (image processing unit 101c) detects the number of frames n of the image of the detection target. Once the number of frames n has been detected, the process proceeds to the next step S203. In step S203, the controller 101 (image processing unit 101c) sets a count value m to 1. Once the count value m has been set, the process proceeds to the next step S204. In step S204, the controller 101 (image processing unit 101c) processes the image of the detection target in the m-th frame. For image processing in the image processing unit 101c, an AI model specified by AI model information INF1 stored in the memory 102 is used. If multiple drive recorders 1 use AI models with different processing contents, multiple types of AI model information INF1 may be stored in the memory 102, and the identification information associated with the feature map corresponding to the hash value calculated in step S102 of Fig. 6 may include the identification information of the AI ​​model. The controller 101 (image processing unit 101c) may use the AI ​​model that matches the identification information of the AI ​​model associated with the feature map corresponding to the hash value calculated in step S102 of Fig. 6. The controller 101 (image processing unit 101c) generates at least a feature map (more specifically, a feature map after pooling) obtained after convolution processing in the convolution layer (k-th convolution layer) Lk for the image of the detection target. After image processing is performed, the process proceeds to the next step S205.

[0078] In step S205, the controller 101 (hash value calculation unit 101a) inputs the feature map (more specifically, the feature map after pooling) obtained from the image of the detection target in the m-th frame into a hash function to calculate a hash value (hash value for detection). This hash function is also a hash function specified by hash function information INF2 stored in the memory 102. Once the hash value for detection is calculated, the process proceeds to the next step, S206.

[0079] In step S206, the controller 101 (comparison unit 101d) inputs the feature map (more specifically, the feature map after pooling) received by the communication unit 103 into a hash function to calculate a hash value (the hash value calculated in step S102 and saved in step S103 for the image to be detected (identified by information for identifying the image of the frame, such as the shooting time)) with the detection hash value, and stores the comparison result in memory 102. After the comparison process and storage process are performed, the process proceeds to the next step, S207.

[0080] In step S207, the controller 101 (image processing unit 101c) determines whether or not the count value m has reached the number of frames n. If the count value m has not reached the number of frames n, the controller 101 (image processing unit 101c) increments the count value m by one (processing in step S208), and then the process returns to step S204. On the other hand, if the count value m has reached the number of frames n, the process proceeds to step S209.

[0081] In step S209, the controller 101 (determination unit 101e) determines that the image to be detected is a tampered image if there is a mismatch in any of the comparison results (n comparison results stored in memory 102) by the comparison unit 101d for the first to nth frames. Note that, taking into consideration the possibility of image data corruption due to noise or the like, the image to be detected may be determined to be a tampered image if the mismatch rate in the comparison results by the comparison unit 101d is equal to or greater than a threshold value (for example, 1%). Once the determination process is performed, the flow shown in FIG. 7 ends.

[0082] 4. Second Embodiment of Drive Recorder Fig. 8 is a diagram illustrating a second embodiment of the drive recorder 1. The drive recorder 1 illustrated in Fig. 8 differs from the drive recorder 1 illustrated in Fig. 2 in that the controller 6 includes a hash value calculation unit 6d, but is otherwise basically the same as the drive recorder 1 illustrated in Fig. 2.

[0083] The hash value calculation unit 6d inputs the feature map (specifically, the feature map after pooling) obtained after the convolution process in the convolution layer (k-th convolution layer) Lk into a hash function to calculate a hash value.

[0084] The communication unit 12 transmits the hash value calculated by the hash value calculation unit 6d to the server 100 via the network N1.

[0085] In the drive recorder 1 shown in Figure 8, the image processing unit 6b applies the original image for the detection target image to the AI ​​model M1 to calculate the original image features, the hash value calculation unit 6d applies the original image features to a hash function to calculate a comparison reference hash value, and the communication unit 12 transmits the comparison reference hash value to the server 100.

[0086] 5. Second embodiment of the server Fig. 9 is a diagram showing a second embodiment of the server 100. The server 100 shown in Fig. 9 constitutes a hash value storage system SYS1 together with the drive recorder 1 shown in Fig. 8. The server 100 shown in Fig. 9 differs from the server 100 shown in Fig. 5 in that the controller 101 includes a detection hash value calculation unit 101f instead of the hash value calculation unit 101a, but is otherwise basically the same as the server 100 shown in Fig. 5.

[0087] The communication unit 103 receives the hash value (comparison reference hash value) calculated by the hash value calculation unit 6d of the drive recorder 1 via the network N1, and the received hash value HV is stored in the memory .

[0088] The hash value calculation unit 101f for detection inputs a feature map (specifically, the feature map after pooling) obtained from the image of the detection target into a hash function to calculate a hash value (hash value for detection). The hash function is a hash function specified by hash function information INF2 stored in the memory 102, and is the same as the hash function used by the hash value calculation unit 6d of the drive recorder 1.

[0089] In the server 100 shown in Figure 9, the image acquisition unit 101b acquires a detection target image, the image processing unit 101c applies an AI model with the same processing content as the AI ​​model M1 to calculate the detection target image features, the detection hash value calculation unit 101f applies the detection target image features to a detection hash function with the same processing content as the hash function used in the hash value calculation unit 6d of the drive recorder 1 to calculate the detection hash value, the comparison unit 101d compares the comparison reference hash value with the detection hash value, and the judgment unit 101e judges whether the detection target image has been tampered with based on the comparison result of the comparison unit 101d.

[0090] In the first embodiment, a feature map (more specifically, a feature map after pooling) is transmitted from the drive recorder 1 shown in Fig. 2 to the server 100 shown in Fig. 5, whereas in the second embodiment, a hash value is transmitted from the drive recorder 1 shown in Fig. 8 to the server 100 shown in Fig. 9. Therefore, in the second embodiment, the size of data transmitted from the drive recorder 1 to the server 100 can be reduced compared to the first embodiment, and the communication load can be reduced.

[0091] <6. Points to note> Various technical features disclosed as embodiments in this specification may be modified in various ways without departing from the spirit of the technical creation. In other words, the above-described embodiments are illustrative in all respects and are not limiting. The technical scope of the present invention is defined by the claims, not by the description of the above-described embodiments, and includes all modifications that fall within the meaning and scope of the claims. Furthermore, the multiple embodiments described in this specification may be combined as appropriate to the extent possible.

[0092] For example, unlike the first and second embodiments described above, the drive recorder 1 may transmit images of each frame periodically transmitted from the camera 2 to the server 100, and the server 100 may calculate a feature map (more specifically, a feature map after pooling) obtained after convolution processing in the convolution layer (k-th convolution layer) Lk of the AI ​​model, and input the calculated feature map (more specifically, a feature map after pooling) to a hash function to calculate a hash value. In this case, the AI ​​model in the server 100 may have a structure different from that of the AI ​​model M1 used in the drive recorder 1.

[0093] Alternatively, for example, instead of the feature map (specifically, the feature map after pooling), the image processing results output from the AI ​​model may be input to a hash function to calculate a hash value, thereby further reducing the processing load for calculating the hash value. [Explanation of symbols]

[0094] 1. Drive recorder 2 Cameras 3. Microphone 4 Control section 5 speakers 6 Controller 6a Image acquisition unit 6b Image processing section 6c Warning processing section 6d Hash value calculation section 7. Memory 8 card slots 9 Timing circuit 10 Acceleration Sensor 11 GPS receiver 12, 103 Communications Department 100 servers 101 Controller 101a Hash value calculation unit 101b Image acquisition unit 101c Image processing unit 101d Comparison section 101e Judgment section 101f Hash value calculation unit for detection 102 memory 103 Communications Department M1 AI model MC1 memory card L1~Ln convolution layers LOUT output layer NW1 In-vehicle network SEN1 Vehicle information sensor SEN2 Vehicle speed sensor SEN3 steering angle sensor SYS1 hash value storage system

Claims

1. A tampering detection method for detecting tampering with a detection target image from an original image, comprising: Applying the original image to a first AI model to calculate original image features; Applying the original image feature quantity to a hash function to calculate a comparison reference hash value; Applying the detection target image to a second AI model having the same processing content as the first AI model to calculate a detection target image feature amount; applying the detection target image feature quantity to a detection hash function having the same processing content as the hash function to calculate a detection hash value; determining whether the data has been tampered with by comparing the comparison reference hash value with the detection hash value; Tamper detection method.

2. Calculate features for the image using an AI model, A hash value calculation method in which the calculated feature amount or the image processing result output from the AI ​​model is input into a hash function to calculate a hash value.

3. The hash value calculation method according to claim 2 , wherein the feature is a feature obtained when a convolution process is performed multiple times by the AI ​​model.

4. The hash value calculation method according to claim 2 , wherein the feature is obtained by thinning out the number of channels from the feature generated by executing a convolution process using the AI ​​model.

5. 5. The hash value calculation method according to claim 2, wherein the image is an image obtained by decompressing an original image using lossy compression.

6. Calculate features for the image using an AI model, An in-vehicle device that inputs the calculated feature amount or the image processing result output from the AI ​​model into a hash function to calculate a hash value.

7. Calculating features for the image using an AI model; inputting the calculated feature amount or the image processing result output from the AI ​​model into a hash function to calculate a hash value; A hash value calculation program that causes a computer to execute the following.

8. an in-vehicle device that applies a first AI model to an original image corresponding to the detection target image to calculate original image feature amounts and transmits the original image feature amounts; a server that receives the original image feature amount, applies the original image feature amount to a hash function to calculate a comparison reference hash value, and stores the comparison reference hash value; A hash value storage system comprising:

9. an in-vehicle device that applies an original image corresponding to the detection target image to a first AI model to calculate original image features, applies the original image features to a hash function to calculate a comparison reference hash value, and transmits the comparison reference hash value; a server that receives the comparison reference hash value and stores the comparison reference hash value; A hash value storage system comprising:

10. The hash value storage system according to claim 8 or claim 9, wherein the server applies the detection target image to a second AI model having the same processing content as the first AI model to calculate detection target image features, applies the detection target image features to a detection hash function having the same processing content as the hash function to calculate a detection hash value, and determines whether tampering has occurred by comparing the comparison reference hash value with the detection hash value.

Images