Equipment data processing method and device, product, equipment and medium

Abstract

The invention discloses an equipment data processing method and device, a product, equipment and a medium. The method comprises the following steps: acquiring a target image acquired by a target camera component; the target camera component is a camera component in the target equipment; in the target image, extracting component imaging features of the target camera component; the component imaging features are used for reflecting non-uniform light response features of the target camera component during imaging; and generating a device fingerprint of the target device based on the component imaging features. By adopting the method and the device, the accuracy and the stability of the device fingerprint generated for the target device can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, product, equipment and medium for processing device data. Background Art

[0002] The device fingerprint of a device is a device feature used to uniquely identify the device. In existing applications, the device fingerprint of a device can usually be generated through some usage information of the device (such as the working information of the device's processor). However, since different devices may have the same or similar working conditions (such as the owners of different devices have the same or similar habits of using the devices), the device fingerprints generated for each device are not very distinguishable. In addition, since the working conditions of the device may change, the device fingerprint of the device may also change, making the device fingerprint of the device unstable and inaccurate. Therefore, how to generate a more accurate and stable device fingerprint of the device has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method, apparatus, product, device and medium for processing device data, which can improve the accuracy and stability of device fingerprints generated for target devices.

[0004] On the one hand, the present application provides a method for processing device data, the method comprising:

[0005] Acquire a target image captured by a target camera component; the target camera component is a camera component in a target device;

[0006] Extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response characteristics of the target camera component during imaging;

[0007] Generate device fingerprints of target devices based on component imaging features.

[0008] On one hand, the present application provides a device for processing device data, the device comprising:

[0009] An acquisition module, used for acquiring a target image captured by a target camera component; the target camera component is a camera component in a target device;

[0010] An extraction module is used to extract component imaging features of a target camera component in a target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging;

[0011] The generation module is used to generate a device fingerprint of the target device based on the component imaging features.

[0012] Optionally, the extraction module extracts the component imaging features of the target camera component in the target image, including:

[0013] Perform signal decomposition processing on the target image to obtain the image noise signal contained in the target image;

[0014] Based on the image noise signal, a component imaging feature of the target imaging component is generated.

[0015] Optionally, the extraction module generates the component imaging feature of the target camera component based on the image noise signal, including:

[0016] The low-frequency signal in the image noise signal is removed to obtain a transition noise signal;

[0017] Based on the transition noise signal, a component imaging signature is generated.

[0018] Optionally, the transition noise signal includes noise signals of the target image in multiple reference directions, and the noise signal in one reference direction has a signal intensity coefficient obtained after performing signal decomposition processing on the target image;

[0019] The extraction module generates component imaging features based on the transition noise signal, including:

[0020] De-noising the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the de-noised signal strength coefficient in each reference direction;

[0021] Based on the denoised signal intensity coefficients in each reference direction, component imaging features are generated.

[0022] Optionally, any one of the multiple reference directions is a target reference direction, and the signal strength coefficient of the noise signal in the target reference direction includes multiple coefficient values;

[0023] The extraction module performs denoising on the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction, including:

[0024] Performing variance estimation processing on a plurality of coefficient values ​​in the signal strength coefficient of the noise signal in the target reference direction to obtain an estimated variance value in the target reference direction;

[0025] Based on the estimated variance value, the signal strength coefficient of the noise signal in the target reference direction is denoised to obtain the denoised signal strength coefficient in the target reference direction.

[0026] Optionally, the extraction module generates a component imaging feature based on the denoised signal intensity coefficient in each reference direction, including:

[0027] Determine the denoised noise signal in each reference direction based on the denoised signal strength coefficient in each reference direction;

[0028] The denoised noise signal in each reference direction is decomposed and inversely transformed to obtain a noise image in each reference direction;

[0029] The noise images in each reference direction are fused to obtain the component imaging features.

[0030] Optionally, the target image is subjected to signal decomposition processing based on a wavelet transform method;

[0031] The denoised noise signal in each reference direction is decomposed and inversely transformed based on an inverse transformation method of a wavelet transformation method.

[0032] Optionally, the extraction module extracts the component imaging features of the target camera component in the target image, including:

[0033] Get the image cropping rule; the image cropping rule is used to define the size and position of the image to be cropped uniformly;

[0034] The target image is cropped according to the image cropping rule to obtain a cropped image of the target image;

[0035] In the cropped image, component imaging features are extracted.

[0036] Optionally, the target device has an associated device type; the apparatus is further configured to:

[0037] Get the device classification network to be trained;

[0038] Call the device classification network to be trained to predict the device type of the target device based on the device fingerprint of the target device;

[0039] Based on the difference between the device type predicted for the target device and the device type associated with the target device, the network parameters of the device classification network to be trained are corrected to obtain a trained device classification network.

[0040] Optionally, the apparatus corrects network parameters of a device classification network to be trained based on a difference between a device type predicted for a target device and a device type associated with the target device to obtain a trained device classification network, including:

[0041] Based on the device type predicted for the target device and the device type associated with the target device, generating a device classification loss for the target device by the device classification network; the device classification loss is used to reflect the difference between the device type predicted for the target device and the device type associated with the target device;

[0042] The network parameters of the device classification network to be trained are corrected based on the device classification loss to obtain a trained device classification network.

[0043] Optionally, there are multiple target devices, one target device has one associated device type, and multiple target devices have multiple associated device types;

[0044] One target device has one predicted device type, and the device type predicted for any target device among the multiple target devices is any one of the multiple device types.

[0045] Optionally, the above device is also used for:

[0046] Obtaining a verification image sent by a verification device; the verification image is obtained by real-time shooting of a camera component contained in the verification device;

[0047] Based on the verification image, generate a device fingerprint of the verification device;

[0048] The trained device classification network is called to predict the target device type of the verification device among multiple device types based on the device fingerprint of the verification device.

[0049] Optionally, each device type in the plurality of device types has an associated device service;

[0050] The above device is also used for:

[0051] If the verification device sends a call request for the target device service before sending the verification image, and the target device service is consistent with the device service associated with the target device type, the target device service is provided to the verification device in response to the call request.

[0052] In one aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a method in one aspect of the present application.

[0053] In one aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0054] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in various optional modes such as the above-mentioned one aspect.

[0055] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and the component imaging features of the target camera component can be extracted in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and further, the device fingerprint of the target device can be generated based on the component imaging features. It can be seen that the method proposed in the present application can generate a device fingerprint of the target device through the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, the device fingerprint of the target device and the component imaging features of the target camera component have a strong correlation, and the device fingerprint of the target device will also be very stable and can be used to accurately identify the target device having the target camera component. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 It is a structural diagram of a network architecture provided by this application;

[0058] Figure 2 This is a schematic diagram of a scenario for generating device fingerprints provided by this application;

[0059] Figure 3 It is a flowchart of a method for processing device data provided by the present application;

[0060] Figure 4 This is a schematic diagram of a variance estimation scenario provided by this application;

[0061] Figure 5 It is a flowchart of a feature extraction method provided by this application;

[0062] Figure 6 It is a scenario diagram of a network training method provided by this application;

[0063] Figure 7 It is a schematic diagram of a network training scenario provided by this application;

[0064] Figure 8 This is a schematic diagram of a process for classifying devices based on device fingerprints provided by this application;

[0065] Fig. 9It is a structural schematic diagram of a device data processing apparatus provided by the present application;

[0066] Fig.10 It is a structural schematic diagram of a computer device provided by this application. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] This application involves artificial intelligence related technologies. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0069] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction system, mechatronics, etc. Among them, the pre-trained model is also called a large model or a basic model. After fine-tuning, it can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0070] This application mainly involves machine learning in artificial intelligence. Among them, machine learning (ML) is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development in deep learning, which integrates the above technologies.

[0071] The machine learning involved in this application mainly refers to how to train a device classification network to identify and classify devices based on the device fingerprint of the device through the device classification network. For details, please refer to the following Figure 6 Related description in the corresponding embodiment.

[0072] The present application also involves blockchain-related technologies. Among them, blockchain is a new application model of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain may include a blockchain underlying platform, a platform product service layer, and an application service layer. The blockchain includes a series of blocks that are connected to each other in the order of their generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the nodes in the blockchain system. In this application, the device fingerprint generated for the device can be stored on the chain to ensure the security of the device fingerprint.

[0073] First of all, it needs to be explained that all data collected in this application (such as images collected by the target camera component, images collected by the verification device through the included camera component, and other related data) are collected with the consent and authorization of the object to which the data belongs (such as a user, organization or enterprise), and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0074] Here, the relevant technical concepts involved in this application are explained:

[0075] Device fingerprint: can be used to uniquely identify the device characteristics or unique device identifier of the device.

[0076] PRNU: Photo Response Non-Uniformity, photo response non-uniformity, also known as non-uniform photo response characteristics, is a noise feature. As the main component of sensor pattern noise, PRNU characteristics are caused by slight differences in the photosensitivity of the imaging device's photosensitive elements due to production process defects caused by differences in the thickness of the silicon coating between sensor pixels, resulting in a fixed error distribution in the entire photosensitive array of the imaging device's photosensitive elements. This fixed error distribution is also implicit in the image captured by the imaging device's photosensitive elements in the form of a multiplicative factor.

[0077] The photosensitive element of the imaging device may be a photosensitive element in a camera assembly (such as a camera in a device) (such as a light-sensitive sensor in a camera).

[0078] See also Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided by this application. Figure 1 As shown, the network architecture may include a server 200 and a terminal device cluster, and the terminal device cluster may include one or more terminal devices, and the number of terminal devices is not limited here. Figure 1 As shown, the multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n; Figure 1 As shown, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all be connected to the server 200 through a network, so that each terminal device can exchange data with the server 200 through the network connection.

[0079] like Figure 1 The server 200 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smart terminal such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, etc. The following takes the communication between the terminal device 1 and the server 200 as an example to describe the specific embodiments of the present application.

[0080] See also Figure 2 , Figure 2 This is a schematic diagram of a scenario for generating device fingerprints provided by this application. Figure 2As shown, the terminal device 1 may include a camera component, which may be a camera in the terminal device 1. The terminal device 1 may be referred to as a target device, and the camera component in the target device may be referred to as a target camera component.

[0081] Optionally, the target camera component may include elements for imaging such as a lens, an anti-aliasing filter, a filter (CFA), and a sensor. The target camera component can shoot a scene, and the captured image passes through the lens, anti-aliasing filter, filter, and sensor in the target camera component in sequence, and finally a target image can be captured. The target image is the photo taken by the target camera component.

[0082] Among them, since the sensor in the target camera assembly has pattern noise during the imaging process, the pattern noise may be PRNU noise, and therefore, the target image obtained by the above shooting may contain the pattern noise.

[0083] The server 200 can obtain the target image (for example, the target image can be taken by using the target camera component when the target camera component is produced and sent to the server 200). Then, the server 200 can extract the imaging feature of the target camera component from the target image. The imaging feature can be the PRNU noise generated by the sensor in the target camera component. The imaging feature can be called the component imaging feature. The component imaging feature can be used to reflect the non-uniform light response feature of the target camera component. The specific process of extracting the component imaging feature from the target image can be referred to as follows: Figure 3 Corresponding to the description in the embodiment.

[0084] Furthermore, the server 200 can generate a device fingerprint of the target device through the above component imaging features. Subsequently, the server 200 can accurately identify and verify the identity of the target device based on the generated device fingerprint of the target device. The specific process can be found in the relevant descriptions of the following embodiments.

[0085] In the present application, since the component imaging feature of the target device is an inherent imaging feature of the target camera component in the target device, an accurate and stable device fingerprint of the target device can be generated through the component imaging feature.

[0086] See also Figure 3 , Figure 3This is a flowchart of a method for processing device data provided by the present application. The execution subject in the embodiment of the present application may be a device for processing device data (which may be referred to as a data processing device), which may be a computer device or a computer device cluster composed of multiple computer devices, which may be a server, a terminal device, or other devices, without limitation. Figure 3 As shown, the method may include:

[0087] Step S101, acquiring a target image captured by a target camera component; the target camera component is a camera component in a target device.

[0088] Optionally, the target device may have a camera component, which may be referred to as a target camera component. The target device may be a terminal device or other device, such as a merchant's cash register, a user's handheld terminal device, or a portable terminal device, etc. The target camera component may be a component used for taking pictures in the target device, such as a camera in the target device, which has a sensor. Optionally, the target device may be an IOT device (Internet of Things device).

[0089] The data processing device can obtain a target image captured (e.g., photographed) by the target camera component. The target image can be any image captured by the target camera component. Optionally, the target image can be a photo taken by the target device, or the target image can be a video frame selected from a video taken by the target camera component. Subsequently, the data processing device can generate a device fingerprint of the target device through the target image.

[0090] Optionally, the target image can be captured by the target camera assembly when the target camera assembly leaves the factory or is assembled into the target device, so that the device fingerprint of the target device can be pre-generated before the target device is put into use, which is subsequently used to identify the target device.

[0091] Alternatively, the target image may be acquired by calling the target camera component in the target device when the target device's device fingerprint is needed to identify the target device after the target device is put into use. Here, calling the target camera component to capture the image may be done with the consent of the user of the target device. For example, a query pop-up window may be popped up so that the user can successfully call the target camera component to capture the image or video after clicking the option of agreeing on the pop-up window interface. The target device may send the captured image to the data processing device.

[0092] Step S102 : extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging.

[0093] Optionally, the data processing device can extract component imaging features of the target camera assembly in the target image. The component imaging features may be PRNU features extracted from the target image. The component imaging features may be used to reflect the non-uniform light response characteristics of the target camera assembly during imaging. The PRNU features may be caused by the uneven thickness of the silicon coating at various positions of the photosensitive element of the target camera assembly. The PRNU features are inherent features of the target camera assembly.

[0094] It can be understood that the PRNU features contained in different images taken by the same camera assembly are the same, and even if there are differences between the PRNU features contained in different images taken by the same camera assembly, the differences are very subtle and can be ignored. Therefore, the PRNU features contained in different images taken by the target camera assembly can be considered to be the same.

[0095] The following describes the specific process of extracting the component imaging features of the target camera component from the target image.

[0096] First, a formula is introduced to explain the PRNU feature. This formula is based on the noise extraction method of wavelet filtering to obtain PRNU noise (i.e., PRNU feature). This formula regards the image as a composite of a noise-free image and a noise image. As shown in the following formula:

[0097] y ij =f ij (x ij +η ij )+c ij +∈ ij (1)

[0098] Wherein, i represents the number of rows, j represents the number of columns, i is a positive integer less than or equal to the total number of rows of the silicon coating of the target camera component, and j is a positive integer less than or equal to the total number of columns of the silicon coating of the target camera component. ij The PRNU multiplicative noise factor used to determine the pixel position at the i-th row and j-th column of the silicon coating of the target camera component, f ij Can be a noise function related to PRNU noise. ij It can represent the overall noise signal of the image defined by the above formula.

[0099] Among them, the size (which can be understood as the area) of the silicon coating of the target camera component is equal to the size of the image captured by the target camera component. Therefore, a pixel position of the silicon coating of the target camera component corresponds to the position of a pixel point of the target image, such as the pixel position at the i-th row and j-th column of the silicon coating of the target camera component corresponds to the position of the pixel point at the i-th row and j-th column in the target image. Therefore, the PRNU multiplicative noise factor of the pixel position at the i-th row and j-th column of the silicon coating of the target camera component can be the PRNU multiplicative noise factor of the pixel point at the i-th row and j-th column in the target image.

[0100] x ij represents the incident light of the sensor (which may be a photosensitive element) of the target camera assembly, η ij It represents shot noise, which is noise associated with PRNU. The shot noise is caused by electron emission non-uniformity (emission non-uniformity caused by uneven thickness of silicon coating). The shot noise can also be called environmental noise.

[0101] c ij Represents dark current noise, which is the noise caused by the current when the target camera component takes an image. It is usually fixed and is a noise characteristic that the target camera component has when it leaves the factory. ∈ ij Indicates additional random noise, which is a relatively small noise added to balance various noises. It can be a fixed value. The additional random noise can be present or absent. Dark current noise and additional random noise are not noises associated with PRNU noise.

[0102] Therefore, in this application, it is necessary to retain f ij (x ij +η ij ) and remove the noise of c ij +∈ ij This part of the noise.

[0103] Based on the principles described above, the specific process of extracting component imaging features of a target camera component from a target image in the present application can be described as follows.

[0104] First, the data processing device can perform signal decomposition processing on the target image to obtain the noise signal (which can be called image noise signal) and non-noise signal contained in the target image. The data processing device can perform signal decomposition processing on the target image using a wavelet transform method (such as fourth-level wavelet transform (4-layer wavelet transform)). Optionally, the wavelet basis of the wavelet transform can be db4 (a wavelet function that can be used to determine the waveform of the wavelet transform) by default, and can also be other wavelet bases according to different actual application scenarios.

[0105] Since the above-mentioned dark current noise and additional random noise are usually the noise of low-frequency signals, the present application can remove the low-frequency signal in the image noise signal obtained by the above decomposition to obtain a transition noise signal, that is, the noise signal obtained after removing the low-frequency signal in the image noise signal can be called a transition noise signal.

[0106] The low-frequency signal may refer to a signal whose frequency is lower than a set frequency threshold, and the low-frequency signal is a signal that appears less frequently in the image noise signal. The frequency threshold may be set according to actual application scenarios.

[0107] The above-mentioned transition noise signal may include multiple parts of noise signals obtained by wavelet transform decomposition, and the multiple parts of noise signals may include noise signals of the target image in multiple directions (which may be called reference directions). Optionally, the multiple reference directions may include horizontal, vertical and diagonal directions.

[0108] In other words, since the low-frequency signal (which may be referred to as the low-frequency component) in the image noise signal has been removed, the transition noise signal may include a high-frequency noise component in the horizontal direction (which may also be referred to as the high-frequency noise signal), a high-frequency noise component in the vertical direction, and a high-frequency noise component in the diagonal direction. It is understandable that the above-mentioned image noise signal may also include noise signals in the above-mentioned multiple reference directions (the noise signal in each reference direction may include a low-frequency noise signal and a high-frequency noise signal).

[0109] It should be noted that the target image may include images on multiple channels (such as three channels including R (red color channel), G (green color channel), and B (blue color channel)) (which can be understood as image components of the target image on multiple channels). Therefore, the above-mentioned process of performing signal decomposition processing on the target image to obtain the image noise signal contained in the target image may include: performing signal decomposition processing (such as wavelet transform processing) on ​​the image components of the target image on each channel respectively, and the image noise signal of the target image on each channel can be obtained. One channel can have one of the above-mentioned image noise signals.

[0110] Subsequently, the component imaging features of the target camera component can be generated based on the image noise signals of the target image on each channel. Since the principles for processing the image components on each channel are the same and independent, the following description will be given by taking the processing process of the image noise signal on any channel as an example.

[0111] Among them, the noise signals in each reference direction contained in the above-mentioned transition noise signal can have a signal strength coefficient, and the signal strength coefficient can be used to reflect the strength of the noise signals in each reference direction. The noise signal in a reference direction can have a signal strength coefficient. The larger the signal strength coefficient is, the greater the strength of the noise signal is. Conversely, the smaller the signal strength coefficient is, the smaller the strength of the noise signal is.

[0112] The signal intensity coefficient of the noise signal in each reference direction in the transition noise signal can also be generated after the target image is subjected to signal decomposition processing. The signal intensity coefficient of the noise signal in each reference direction can be the wavelet coefficient in each reference direction obtained after the target image is subjected to signal decomposition processing, that is, the signal intensity coefficient in one reference direction can be the wavelet coefficient decomposed in the reference direction. The wavelet coefficient of the noise signal in one reference direction can also be used to reflect the similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform. The larger the wavelet coefficient, the greater the similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform. Conversely, the smaller the wavelet coefficient, the smaller the similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform.

[0113] The data processing device can perform denoising on the signal strength coefficients of the noise signals in the above-obtained multiple reference directions to obtain the denoised signal strength coefficients in each reference direction. Furthermore, the data processing device can generate the above-mentioned component imaging features through the denoised signal strength coefficients in the multiple reference directions of the transition noise signal.

[0114] Optionally, the process of the data processing device performing denoising on the signal strength coefficients of the noise signals in the above-mentioned multiple reference directions may include:

[0115] Among them, any reference direction among the above-mentioned multiple reference directions can be called the target reference direction. The following is an example of denoising the signal strength coefficient of the noise signal of a transition noise signal on a channel (can be any one) in a reference direction.

[0116] Optionally, the signal strength coefficient of the noise signal in the target reference direction (i.e., the wavelet coefficient) can be a vector (which can be called a coefficient vector), which can contain multiple values ​​(i.e., multiple elements). The multiple values ​​can be referred to as multiple coefficient values ​​contained in the signal strength coefficient (i.e., the values ​​of the coefficients of the wavelet transform). In other words, the signal strength coefficient of the noise signal in the target reference direction can contain multiple coefficient values.

[0117] The data processing device can perform variance estimation (which can be a local variance estimation) on multiple coefficient values ​​in the signal strength coefficient of the noise signal in the target reference direction (which can be referred to as the signal strength coefficient in the target reference direction) to obtain an estimated variance value in the target reference direction, wherein the local variance estimation can be achieved through an initial standard deviation, and the default value of the initial standard deviation can be 5 (or other values).

[0118] The process of estimating the local variance of the signal strength coefficient in the target reference direction may include:

[0119] The data processing device can obtain a target window (belonging to a sliding window), which is a window used to perform local variance estimation on the signal strength coefficient in the above-mentioned target reference direction. The target window has a set window size (which can be set according to the actual application scenario, such as 3, 5, 7 or 9, etc.). Furthermore, through the target window, multiple coefficient values ​​of the signal strength coefficient in the target reference direction can be slidingly estimated, and the sliding step size can also be determined according to the actual application scenario, such as 1 or 3, etc.

[0120] Furthermore, each sliding of the target window can select coefficient values ​​of the signal strength coefficient in the target reference direction, the number of which is equal to the above-mentioned window size. The target window can slide multiple times according to the step size, so that each coefficient value in the signal strength coefficient in the target reference direction can be selected eventually. The data processing device can perform variance estimation on the coefficient values ​​selected for the starting position of the target window and the position to which it slides each time, so as to obtain the estimated variance values ​​of the coefficient values ​​selected at each position of the target window. One position can correspond to one estimated variance value, and further, the smallest estimated variance value among multiple estimated variance values ​​corresponding to multiple positions can be used as the estimated variance value in the target reference direction.

[0121] For example, if the signal strength coefficient in the target reference direction has 9 coefficient values ​​(coefficient value 1 to coefficient value 9 in sequence), the size of the target window is 3, and the step length is 3, then first when the target window is at the starting position, the coefficient values ​​1 to coefficient value 3 among the 9 coefficient values ​​can be selected through the target window, and the variance of the coefficient values ​​1 to coefficient value 3 can be estimated to obtain an estimated variance value of 1, and then the target window is slid to the right by 3 coefficient values ​​(because the step length is 3), so that the target window can select coefficient values ​​4 to coefficient value 6 among the 9 coefficient values. , and the variance of the coefficient value 4 to the coefficient value 6 can be estimated to obtain the estimated variance value 2. Furthermore, the data processing device can slide the target window to the right by 3 coefficient values ​​(because the step size is 3), so that the target window can select the coefficient value 7 to the coefficient value 9 among the 9 coefficient values, and the variance of the coefficient value 7 to the coefficient value 9 can be estimated to obtain the estimated variance value. Finally, the smallest estimated variance value among the estimated variance value 1, the estimated variance value 2 and the estimated variance value 3 obtained by the above estimation can be used as the estimated variance value in the target reference direction.

[0122] See also Figure 4 , Figure 4 This is a schematic diagram of a variance estimation scenario provided by this application. Figure 4 As shown, the signal strength coefficient in the target reference direction can have 9 coefficient values ​​from 1 to 9, and the sliding window here can be the above-mentioned target window. Here, the step size of the target window can be 2 and the window size can be 3.

[0123] Therefore, when the sliding window is at the starting position, coefficient values ​​1 to 3 of the signal strength coefficients in the target reference direction may be selected, and variances of the coefficient values ​​1 to 3 may be estimated to obtain an estimated variance value 1.

[0124] Furthermore, by sliding the sliding window to the right by 2 coefficient values, coefficient values ​​3 to 5 of the signal strength coefficient in the target reference direction can be selected, and variance estimation is performed on the coefficient values ​​3 to 5 to obtain an estimated variance value 2.

[0125] Next, the sliding window is slid rightward again by 2 coefficient values, and coefficient values ​​5 to 7 of the signal strength coefficient in the target reference direction can be selected. The variance of the coefficient values ​​5 to 7 is estimated, and an estimated variance value of 3 can be obtained.

[0126] Furthermore, the sliding window is slid rightward again by 2 coefficient values, and coefficient values ​​7 to 9 in the signal strength coefficient in the target reference direction can be selected, and variance estimation is performed on the coefficient values ​​7 to 9, and an estimated variance value 4 can be obtained. So far, the coefficient values ​​in the signal strength coefficient in the target reference direction have all been subjected to variance estimation.

[0127] The data processing device may take the minimum value among the estimated variance value 1, the estimated variance value 2, the estimated variance value 3, the estimated variance value 4 and the estimated variance value 5 obtained above as the estimated variance value in the target reference direction.

[0128] After obtaining the estimated variance value in the upper target reference direction, the signal strength coefficient of the noise signal in the above target reference direction can be used for denoising, and the denoised signal strength coefficient in the target reference direction can be obtained. Optionally, the process may include: the signal strength coefficient in the target reference direction and the estimated variance value can be input into a filter (such as a Wiener filter) for coefficient denoising, and then the filter can perform denoising on the signal strength coefficient in the target reference direction through the estimated variance value to obtain the denoised signal strength coefficient in the target reference direction.

[0129] It can be understood that the signal strength coefficient in the target reference direction can be filtered (i.e., denoised) by the Wiener filter to obtain the denoised signal strength coefficient, and the variance value of the denoised signal strength coefficient can be made close to the input estimated variance value to filter out the interference signal in the signal strength coefficient in the target reference direction and obtain a signal strength coefficient that tends to be stable.

[0130] The data processing device can perform denoising on the signal strength coefficients of the noise signals in each reference direction in the transition noise signals of each channel through the principles described above, and thus obtain the denoised signal strength coefficients of the noise signals in each reference direction in the transition noise signals of each channel.

[0131] By the above denoising process of the signal strength coefficient in each reference direction, the coefficient values ​​with more sudden changes (such as sudden high or sudden low, etc.) in the signal strength coefficient in each reference direction can be removed to obtain a more accurate and stable signal strength coefficient. It can be understood that by the process of denoising the signal strength coefficient in each reference direction, the noise X of the incident light can also be reduced. ij and shot noise η ij The noise signal with large mutation is removed, so that a more stable and stable component imaging feature can be obtained later (that is, the denoising process of the signal intensity coefficient can be understood as the denoising process of the initial component imaging feature).

[0132] Furthermore, through the denoised signal strength coefficients in the above-mentioned reference directions (including the denoised signal strength coefficients in all channels), the denoised noise signal in each reference direction can be obtained. For example, the denoised signal strength coefficient can be combined with a wavelet (a basic element of wavelet transform, used to represent a signal) to generate a denoised noise signal. The wavelet can be understood as a note in an audio scene, and the denoised signal strength coefficient can be understood as the pitch of the note in the audio scene.

[0133] Optionally, the data processing device can perform signal decomposition and inverse transformation (such as the inverse transformation method of the wavelet transform method, i.e., inverse wavelet transform) on the denoised noise signals in the above-mentioned respective reference directions, so as to generate a noise image in each reference direction, and then perform fusion processing on the noise images in multiple reference directions to obtain the final component imaging feature, which can be an image, and the component imaging feature can be a PRNU feature image. That is, the denoised noise signals in each reference direction can be decomposed and inversely transformed by the inverse wavelet transform method.

[0134] Or, optionally, the data processing device may also first perform signal fusion (such as superposition) on the denoised noise signals in the above-mentioned reference directions to generate a fused denoised signal. Then, the data processing device may perform signal decomposition and inverse transformation (such as an inverse transformation method of the wavelet transform method, i.e., inverse wavelet transform) on the fused denoised signal to generate the final component imaging feature.

[0135] The component imaging features generated by the above two methods are the same. How to generate the component imaging features by using the denoised signals in the above multiple reference directions can be determined according to the actual application scenario.

[0136] Optionally, according to the above, the noise images in each reference direction obtained by directly decomposing and inversely transforming the denoised noise signals in each reference direction and fusing them can be used as the component imaging feature of the target device.

[0137] Alternatively, the present application may also use the noise images in each reference direction obtained by decomposing and inversely transforming the denoised noise signal in each reference direction as the initial component imaging feature of the target device. In order to further remove the noise other than the PRNU feature in the initial component imaging feature, the present application may also perform noise filtering processing (such as zero mean filtering processing) on ​​the initial component imaging feature (belonging to the image) to obtain a more accurate component imaging feature of the target device.

[0138] Optionally, since the sizes of images captured by different camera components may be different, in order to adopt a unified method to generate component imaging features of various camera components, the present application may adopt a unified image cropping rule to uniformly crop the images captured by the camera components, so as to generate component imaging features of the camera components through the cropped images.

[0139] Therefore, the present application can also obtain the image cropping rule, and can use the image cropping rule to perform cropping processing on the above-mentioned target image (the images of the target image on the three channels of R channel, G channel and B channel can be cropped respectively) to obtain a cropped image of the target image (which can include a cropped image of the target image on the R channel, a cropped image on the G channel and a cropped image on the B channel), and then, the component imaging characteristics of the above-mentioned target camera component can be extracted from the cropped image, such as the above-mentioned process of performing signal decomposition processing on the target image to obtain the image noise signal contained in the target image, which can be the process of performing signal decomposition processing on the cropped image of the target image to obtain the image noise signal, that is, the image noise signal can be the noise signal of the cropped image of the target image.

[0140] Optionally, the image cropping rule can be used to define the size and position of uniformly cropping the image. The size of the uniform cropping can be set according to the actual application scenario, such as the size of (224, 224) (the dimension of the cropped image of the target image can be (224, 224, 3), indicating a cropped image with a dimension of (224, 224) on three channels). The position of the uniform cropping can also be set according to the actual application scenario, such as the position of the uniform cropping can be the center of the image. Therefore, the cropped image of the target image obtained by cropping can be an image of the uniform cropping size at the center of the target image.

[0141] The component imaging features generated above can be used to reflect the noise pattern when the target device takes images.

[0142] See also Figure 5 , Figure 5 This is a flow chart of a feature extraction method provided by this application. Figure 5 As shown, the process may include: 1. The data processing device may acquire an RGB image (an image with three color channels), and the RGB image may be a cropped image of the above-mentioned target image.

[0143] 2. The data processing device can perform a single-channel (any of the three color channels) 4-level (4-layer) wavelet transform on the RGB image. The 4-level wavelet transform can refer to a multi-dimensional (here, 4-dimensional) wavelet transform on the RGB image. The wavelet bases of wavelet transforms of different dimensions can be different. Therefore, after performing wavelet transforms of different dimensions on the signal, different signal strength coefficients of the signal can be decomposed. The principle of performing wavelet transform on the signal in each dimension is the same and independent. By performing multi-dimensional wavelet transform on the signal (such as the above-mentioned RGB image), the signal can be decomposed and characterized in multiple dimensions, and a more accurate PRNU feature of the signal can be obtained later.

[0144] 3. Perform local variance estimation and Wiener filtering on the horizontal high-frequency component of a single level (single layer refers to any layer in the four-layer wavelet transform) in the above four-layer wavelet transform (such as the noise signal in the horizontal direction of the above transition noise signal) to obtain the signal strength coefficient after denoising in the horizontal direction of the single level.

[0145] 4. Perform local variance estimation and Wiener filtering on the vertical high-frequency component of the single level (such as the noise signal in the vertical direction of the transition noise signal) to obtain the signal strength coefficient after denoising in the vertical direction of the single level.

[0146] 5. Perform local variance estimation and Wiener filtering on the diagonal high-frequency component of the single level (such as the noise signal in the diagonal direction of the transition noise signal) to obtain the signal strength coefficient after denoising in the diagonal direction of the single level.

[0147] 6. Determine whether the noise signals (such as transition noise signals) on the above 4 levels have been processed, that is, whether the denoised signal strength coefficients in each reference direction of each level have been obtained. If they have been processed, the following step 7 can be executed. If they have not been processed, the above step 3 can be executed again to denoise the signal strength coefficients of the noise signals in each reference direction on the next level (next layer).

[0148] 7. Perform inverse wavelet transform and zero mean filtering on the denoised noise signal in each reference direction on the above single channel (such as any one of the above three color channels) to generate a noise image on the single channel.

[0149] 8. Determine whether the noise images on the above three color channels have been obtained. If the noise images on the above three color channels have been obtained, the following step 9 can be executed. If the noise images on the above three color channels have not been obtained, the above step 2 can be executed again, and then the noise image on the next channel can be obtained according to the above process.

[0150] 9. The final PRNU feature image (i.e., component imaging feature) can be generated by the noise images on the three color channels. For example, the noise images on multiple reference directions on each color channel can be fused separately to obtain the fused noise images on each color channel. The final component imaging feature can include the fused noise images on each color channel.

[0151] Step S103: Generate a device fingerprint of the target device based on the component imaging features.

[0152] Optionally, the data processing device may adopt any of the following methods to generate a device fingerprint of the target device by using the component imaging features of the target camera component extracted from the target image.

[0153] The data processing device can directly image the features of the component as the device fingerprint of the target device.

[0154] The data processing device may obtain local features of the component imaging features as the device fingerprint of the target device. For example, the data processing device may capture a local image from the component imaging features as the device fingerprint of the target device.

[0155] The data processing device can also integrate some device attribute information of the target device itself into the component imaging feature to generate a device fingerprint of the target device. For example, the device attribute information of the target device can include the hardware address (mac address), device identification, or device type of the target device, etc., and the attribute image of the target device can be generated through the device attribute information of the target device. For example, the attribute image can contain text of the attribute information of the target device, and the size of the attribute image can be the same as the size of the component imaging feature. Then, the data processing device can superimpose the attribute image and the component imaging feature of the target camera component (such as adding the pixel values ​​at the same pixel position) to generate the device fingerprint of the target device.

[0156] Through the above-mentioned method of the present application, a unique and stable device fingerprint of the target device can be generated through the imaging characteristics of the target camera component in the target device (such as PRNU characteristics), and the device fingerprint can be subsequently applied in any scenario where the target device needs to be identified, such as in the scenario where the target device is identified and verified through the device fingerprint.

[0157] The above-mentioned device fingerprint generation method of the present application, firstly, since the collection of images taken by the device is very convenient, the calculation amount of extracting PRNU features from the target image is also small, therefore, the device fingerprint of the device can be generated more conveniently; secondly, since the PRNU features of various camera components are inherent and unique, the device fingerprint generated by the PRNU features can make the distinction between different types of camera components highly accurate; thirdly, since the PRNU features are strongly associated with the photosensitive elements of the camera components, the stability of the device fingerprint generated by the PRNU features can also be better.

[0158] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and the component imaging features of the target camera component can be extracted in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and further, the device fingerprint of the target device can be generated based on the component imaging features. It can be seen that the method proposed in the present application can generate a device fingerprint of the target device through the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, the device fingerprint of the target device and the component imaging features of the target camera component have a strong correlation, and the device fingerprint of the target device will also be very stable and can be used to accurately identify the target device having the target camera component.

[0159] See also Figure 6 , Figure 6 Schematic diagram of a network training method provided by this application. Figure 6 As shown, the method may include:

[0160] Step S201, obtaining a device classification network to be trained.

[0161] Optionally, the data processing device may acquire a device classification network to be trained, where the device classification network to be trained may be a network for classifying devices based on device fingerprints of the devices.

[0162] Optionally, the device classification network to be trained in the present application may be SOTA (object recognition) EfficientNet-B4 (an efficient convolutional neural network).

[0163] Step S202: calling a device classification network to predict the device type of the target device based on the device fingerprint of the target device.

[0164] Optionally, there are multiple target devices, and one target device may have an associated device type. The device type associated with the target device refers to the actual device type of the target device (i.e., the real device type). The multiple target devices may have multiple device types in total. The device classification network to be trained can be classified and trained through the target devices of the multiple device types, so that the device classification network to be trained can subsequently identify the multiple device types.

[0165] Optionally, in order to enable the device classification network to be trained to learn the features of the multiple device types more balanced and accurately, the number of target devices of the above-mentioned various device types can be relatively uniform, such as the number of target devices of each device type can be the same, or the absolute value of the difference between the numbers of any two target devices is less than or equal to the target value, and the target value can be a smaller value (can be a positive integer).

[0166] The data processing device can call the device classification network to be trained, and predict the device type of each target device through the device fingerprint of each target device (the device fingerprint obtained through the above-mentioned component imaging features). A target device can have a predicted device type, and the device type predicted for any target device is any one of the multiple device types of the above-mentioned multiple target devices.

[0167] Optionally, the data processing device calls the device classification network to be trained, and the predicted device type of the target device can be represented by a vector (which can be a probability distribution, which can be called a predicted classification probability). The vector can include the probability that the predicted device type of the target device is each of the above-mentioned multiple device types. The device type predicted for the target device can be the device type corresponding to the largest probability in the vector.

[0168] Step S203: based on the difference between the device type predicted for the target device and the device type associated with the target device, modify the network parameters of the device classification network to obtain a trained device classification network.

[0169] Optionally, the data processing device may modify the network parameters of the device classification network to be trained based on the difference between the device type predicted for the target device and the device type associated with the target device. The process may include:

[0170] The data processing device may generate a device classification loss for the target device (which may include the sum of classification losses for each target device) of the device classification network by comparing the device type predicted for the target device with the device type associated with the target device. The device classification loss may be used to indicate the difference between the device type predicted for the target device and the device type associated with the target device. The greater the device classification loss, the greater the difference between the device type predicted for the target device and the device type associated with the target device. Conversely, the smaller the device classification loss, the smaller the difference between the device type predicted for the target device and the device type associated with the target device.

[0171] For example, the device type associated with the target device may also be represented by a vector (which may be referred to as the true classification probability), in which only the probability value corresponding to the device type associated with the target device may be 1, and the probability values ​​corresponding to other device types may be 0. Therefore, optionally, the classification loss for a target device may be the cross entropy loss between the true classification probability and the predicted classification probability of the target device for the above-mentioned multiple device types.

[0172] The data processing device can correct the network parameters of the device classification network to be trained by the device classification loss for the target device obtained above. After the network parameters of the device classification network to be trained are corrected, a trained device classification network can be obtained. The goal of correcting the network parameters of the device classification network to be trained is to make the device classification loss approach the minimum value (e.g., approach 0).

[0173] Optionally, completion of network parameter correction of the device classification network to be trained may refer to correcting the network parameters of the device classification network to be trained to a convergence state, or, refers to the number of iterative training of the network parameters of the device classification network to be trained according to the above principles reaching (greater than or equal to) a threshold number.

[0174] See also Figure 7 , Figure 7 This is a schematic diagram of a network training scenario provided by this application. Figure 7 As shown, the data processing device can input the device fingerprint of the target device into the device classification network to be trained, and the device classification network to be trained can predict the device type of the target device based on the input device fingerprint.

[0175] Furthermore, the data processing device can generate a classification loss for the target device (i.e., the above-mentioned device classification loss) of the device classification network to be trained by the difference between the device type predicted for the target device and the device type associated with the target device. Furthermore, the data processing device can back-propagate the device classification loss to the device classification network to be trained, and the network parameters of the device classification network to be trained can be corrected by the device classification loss, thereby obtaining a trained device classification network.

[0176] The trained device classification network can be used to classify the device type of the device based on the device fingerprint of the device. Optionally, the camera components in the target devices of the same device type are usually produced in batches based on the same method, and the camera components generated in the batch usually have the same imaging features. For example, the camera components generated in the batch can be the camera components in a batch of cash registers.

[0177] The following example describes the process of verifying and identifying a device using the trained device classification network.

[0178] The data processing device can obtain the verification image sent by the verification device, which can be a front-end device, such as a cash register, etc. The verification image can be an image captured in real time by the camera component contained in the verification device, such as the verification image can be an image captured in real time by the data processing device when the verification device requests the data processing device to call a real-time target service (which can be any service, which can be determined according to the actual application scenario, such as a device service for the entire verification device or a service in an application in the verification device), or after the verification device requests the data processing device to call a real-time target service, or an image captured in real time by the verification device actively and sent to the data processing device.

[0179] For example, if the verification device is a cash register, the verification image may be an image captured in real time by the verification device after the cash register initiates a request to the data processing device to call a payment service, or when the cash register initiates a request to the data processing device to call a payment service (which may belong to the above-mentioned target device). The verification device may have a program capable of capturing images in real time, and the data processing device may also have a specific interface for receiving images captured in real time. Therefore, the data processing device can safely and accurately receive the verification image captured in real time by the verification device through the interface, which can ensure that the received verification image is captured in real time by the verification device.

[0180] The data processing device can generate a device fingerprint of the verification device through the received verification image according to the same principle as generating the device fingerprint of the target device.

[0181] Furthermore, the data processing device can call the above-mentioned trained device classification network, and predict the device type of the verification device among multiple device types through the generated device fingerprint of the verification device. The device type with the highest probability predicted for the verification device and greater than the probability threshold among the multiple device types can be used as the target device type predicted for the verification device.

[0182] Each of the above-mentioned multiple device types has an associated device service. A device service associated with a device type may refer to a device service that a target device of the device type can call, that is, the target device may have the authority to call the device service associated with the device type.

[0183] Therefore, if the verification device sends a call request for the target device service before sending the above-mentioned verification image, or sends a call request for the target device service when sending the above-mentioned verification image, and the requested target device service is consistent with the device service associated with the target device type (i.e., the same), then the data processing device can respond to the call request and provide the target device service to the verification device, such as providing payment collection services to the verification device.

[0184] Through the above process, it can be understood that even in some special cases, such as when the device fingerprint of the target device is leaked, the method provided by the present application can also be used to securely and accurately identify the device type (device identity) of the device through the real-time captured image, because the data processing device does not directly receive the device fingerprint, that is, it does not directly verify the received device fingerprint, but receives the real-time captured image, and verifies the device fingerprint generated based on the image. Even if the device fingerprint of the target device is leaked, the malicious actor cannot add the PRNU feature unique to the target device to other images and then give them to the data processing device. Verifying the real-time captured image further isolates the possibility of the malicious actor forging a local image containing the PRNU feature unique to the target device and giving it to the data processing device to skip the identity identification of the target device. Therefore, by using the method provided by the present application, the identity of the device containing the target camera component can be securely and accurately identified through the component imaging feature of the target camera component.

[0185] By adopting the above-mentioned method of the present application, the device classification network can be called to generate a device fingerprint based on the imaging features of the camera component in the device to accurately and automatically identify the identity (i.e., type) of the device. Then, subsequent business processing can be performed on the verification device based on the identification result (such as related processing of device services associated with the device type of the verification device).

[0186] See also Figure 8 , Figure 8 This is a flow chart of device classification based on device fingerprints provided by this application. Figure 8 The corresponding embodiments describe the overall solution process of this application, such as Figure 8 As shown, the process may include:

[0187] S1: The data processing device may collect device videos, which may be videos shot (ie, acquired) by calling a target camera component included in the target device.

[0188] S2: The data processing device can sample and extract the device video, that is, sample (can be random sampling, can also be uniform sampling (such as sampling one video frame every same number of video frames)) multiple video frames from the device video for subsequent network training, and the sampled video frames can be used as the above-mentioned target images.

[0189] S3: The data processing device may perform image cropping processing on the sampled target image, such as performing cropping processing according to the above-mentioned image cropping rules to obtain a cropped image of the target image.

[0190] S4: The data processing device may extract the PRNU feature (ie, component imaging feature) of the target camera component of the target device from the cropped image of the target image, and may generate a device fingerprint of the target device based on the PRNU feature.

[0191] Among them, step S4 may include steps S41 to S44: Step S41, the data processing device may perform multi-scale (such as the above-mentioned 4-level) wavelet transform on the cropped image of the target image. Steps S42 to S43, the data processing device may perform local variance estimation and Wiener filtering on the result of the multi-scale wavelet transform to obtain the denoised noise signal of the cropped image. Step S44, the data processing device may perform inverse wavelet transform processing on the denoised noise signal to obtain the PRNU feature of the target camera component in the target device.

[0192] S5: The data processing device may train a neural network using the device fingerprint of the target device generated above, such as training the device classification network to be trained to obtain a trained device classification network.

[0193] S6: The data processing device can perform network prediction through the trained device classification network, such as predicting the device type of the verification device through the device fingerprint of the verification device.

[0194] S7: The data processing device can identify the verification device by using the trained device classification network to predict the device type of the verification device, such as identifying whether the verification device is a device with the calling authority for the currently initiated target service, and then obtain an identification result for the verification. If the identification result is used to indicate that the verification device is a device with the calling authority for the currently initiated target service, the data processing device can provide the target service to the verification device.

[0195] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of a device data processing apparatus provided by this application. Fig. 9 As shown, the device 1 for processing device data may include: an acquisition module 11 , an extraction module 12 and a generation module 13 .

[0196] An acquisition module 11 is used to acquire a target image captured by a target camera component; the target camera component is a camera component in a target device;

[0197] The extraction module 12 is used to extract the component imaging characteristics of the target camera component in the target image; the component imaging characteristics are used to reflect the non-uniform light response characteristics of the target camera component during imaging;

[0198] The generating module 13 is used to generate a device fingerprint of the target device based on the component imaging feature.

[0199] Optionally, the extraction module 12 extracts the component imaging features of the target camera component in the target image by:

[0200] Perform signal decomposition processing on the target image to obtain the image noise signal contained in the target image;

[0201] Based on the image noise signal, a component imaging feature of the target imaging component is generated.

[0202] Optionally, the extraction module 12 generates the component imaging feature of the target camera component based on the image noise signal, including:

[0203] The low-frequency signal in the image noise signal is removed to obtain a transition noise signal;

[0204] Based on the transition noise signal, a component imaging signature is generated.

[0205] Optionally, the transition noise signal includes noise signals of the target image in multiple reference directions, and the noise signal in one reference direction has a signal intensity coefficient obtained after performing signal decomposition processing on the target image;

[0206] The extraction module 12 generates the component imaging feature based on the transition noise signal, including:

[0207] De-noising the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the de-noised signal strength coefficient in each reference direction;

[0208] Based on the denoised signal intensity coefficients in each reference direction, component imaging features are generated.

[0209] Optionally, any one of the multiple reference directions is a target reference direction, and the signal strength coefficient of the noise signal in the target reference direction includes multiple coefficient values;

[0210] The extraction module 12 performs denoising on the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction, including:

[0211] Performing variance estimation processing on a plurality of coefficient values ​​in the signal strength coefficient of the noise signal in the target reference direction to obtain an estimated variance value in the target reference direction;

[0212] Based on the estimated variance value, the signal strength coefficient of the noise signal in the target reference direction is denoised to obtain the denoised signal strength coefficient in the target reference direction.

[0213] Optionally, the extraction module 12 generates the component imaging feature based on the denoised signal intensity coefficient in each reference direction, including:

[0214] Determine the denoised noise signal in each reference direction based on the denoised signal strength coefficient in each reference direction;

[0215] The denoised noise signal in each reference direction is decomposed and inversely transformed to obtain a noise image in each reference direction;

[0216] The noise images in each reference direction are fused to obtain the component imaging features.

[0217] Optionally, the target image is subjected to signal decomposition processing based on a wavelet transform method;

[0218] The denoised noise signal in each reference direction is decomposed and inversely transformed based on an inverse transformation method of a wavelet transformation method.

[0219] Optionally, the extraction module 12 extracts the component imaging features of the target camera component in the target image by:

[0220] Get the image cropping rule; the image cropping rule is used to define the size and position of the image to be cropped uniformly;

[0221] The target image is cropped according to the image cropping rule to obtain a cropped image of the target image;

[0222] In the cropped image, component imaging features are extracted.

[0223] Optionally, the target device has an associated device type; the apparatus 1 is further used for:

[0224] Obtain the device classification network to be trained;

[0225] Call the device classification network to be trained to predict the device type of the target device based on the device fingerprint of the target device;

[0226] Based on the difference between the device type predicted for the target device and the device type associated with the target device, the network parameters of the device classification network to be trained are corrected to obtain a trained device classification network.

[0227] Optionally, the apparatus 1 corrects network parameters of the device classification network to be trained based on the difference between the device type predicted for the target device and the device type associated with the target device to obtain a trained device classification network, including:

[0228] Based on the device type predicted for the target device and the device type associated with the target device, generating a device classification loss for the target device by the device classification network; the device classification loss is used to reflect the difference between the device type predicted for the target device and the device type associated with the target device;

[0229] The network parameters of the device classification network to be trained are corrected based on the device classification loss to obtain a trained device classification network.

[0230] Optionally, there are multiple target devices, one target device has one associated device type, and multiple target devices have multiple associated device types;

[0231] One target device has one predicted device type, and the device type predicted for any target device among the multiple target devices is any one of the multiple device types.

[0232] Optionally, the device 1 is also used for:

[0233] Obtaining a verification image sent by a verification device; the verification image is obtained by real-time shooting of a camera component contained in the verification device;

[0234] Based on the verification image, generate a device fingerprint of the verification device;

[0235] The trained device classification network is called to predict the target device type of the verification device among multiple device types based on the device fingerprint of the verification device.

[0236] Optionally, each device type in the plurality of device types has an associated device service;

[0237] The above device 1 is also used for:

[0238] If the verification device sends a call request for the target device service before sending the verification image, and the target device service is consistent with the device service associated with the target device type, the target device service is provided to the verification device in response to the call request.

[0239] According to one embodiment of the present application, Figure 3 The steps involved in the device data processing method shown can be Fig. 9 The device data processing device 1 shown in the figure is executed by each module. For example, Figure 3 The step S101 shown in FIG. 1 can be performed by Fig. 9 The acquisition module 11 in is executed, Figure 3 The step S102 shown in FIG. 1 can be performed by Fig. 9 The extraction module 12 is executed; Figure 3 The step S103 shown in FIG. 1 can be performed by Fig. 9 The generation module 13 in is executed.

[0240] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and the component imaging features of the target camera component can be extracted in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and further, the device fingerprint of the target device can be generated based on the component imaging features. It can be seen that the device proposed in the present application can generate a device fingerprint of the target device through the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, the device fingerprint of the target device and the component imaging features of the target camera component have a strong correlation, and the device fingerprint of the target device will also be very stable and can be used to accurately identify the target device having the target camera component.

[0241] According to one embodiment of the present application, Fig. 9The various modules in the device data processing device 1 shown can be individually or all combined into one or several units to constitute, or one (some) of the units can be further divided into multiple functionally smaller sub-units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In actual applications, the functions of one module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the device data processing device 1 may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0242] According to one embodiment of the present application, a computer program capable of executing the steps involved in the corresponding methods shown in the embodiments of the present application can be run on a general-purpose computer device (the computer device may include processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM)) to construct the following. Fig. 9 The device data processing apparatus 1 shown in FIG. The computer program may be recorded on a computer-readable recording medium, and may be loaded into the computer device through the computer-readable recording medium and run therein.

[0243] See also Fig.10 , Fig.10 Schematic diagram of the structure of a computer device provided by this application. Fig.10 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Fig.10 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0244] exist Fig.10In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0245] Acquire a target image captured by a target camera component; the target camera component is a camera component in a target device;

[0246] Extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response characteristics of the target camera component during imaging;

[0247] Generate device fingerprints of target devices based on component imaging features.

[0248] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the description of the above-mentioned device data processing method in each embodiment of the present application, and can also execute the above-mentioned Fig. 9 The description of the device data processing apparatus 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0249] In addition, it should be pointed out here that: the present application also provides a computer-readable storage medium, and a computer program is stored in the computer-readable storage medium. When the processor executes the computer program, it can execute the description of the method for processing device data in each embodiment of the present application, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application.

[0250] As an example, the above-mentioned computer program may be deployed on one computer device for execution, or deployed on multiple computer devices located at one location for execution, or executed on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0251] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0252] The present application provides a computer program product, which includes a computer program, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the description of the processing method of the above-mentioned device data in each embodiment of the present application, so it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application.

[0253] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0254] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0255] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for processing device data, characterized in that: The method comprises: Acquire a target image captured by a target camera component; the target camera component is a camera component in a target device; Extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; Based on the component imaging features, a device fingerprint of the target device is generated.

2. The method according to claim 1, characterized in that Extracting component imaging features of the target camera component in the target image includes: Performing signal decomposition processing on the target image to obtain an image noise signal contained in the target image; Based on the image noise signal, the component imaging feature of the target imaging component is generated.

3. The method according to claim 2, characterized in that The generating the component imaging feature of the target camera component based on the image noise signal comprises: Removing the low-frequency signal from the image noise signal to obtain a transition noise signal; Based on the transition noise signal, the component imaging feature is generated.

4. The method according to claim 3, characterized in that The transition noise signal includes noise signals of the target image in multiple reference directions, and the noise signal in one reference direction has a signal intensity coefficient obtained by performing signal decomposition processing on the target image; The step of generating the component imaging feature based on the transition noise signal comprises: Performing denoising processing on the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction; The component imaging feature is generated based on the de-noised signal intensity coefficient in each reference direction.

5. The method according to claim 4, characterized in that Any one of the multiple reference directions is a target reference direction, and the signal strength coefficient of the noise signal in the target reference direction includes multiple coefficient values; The denoising process is performed on the signal strength coefficient of the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction, including: Performing variance estimation processing on a plurality of coefficient values ​​in the signal strength coefficient of the noise signal in the target reference direction to obtain an estimated variance value in the target reference direction; Based on the estimated variance value, a signal strength coefficient of the noise signal in the target reference direction is denoised to obtain a denoised signal strength coefficient in the target reference direction.

6. The method according to claim 4, characterized in that The generating the component imaging feature based on the denoised signal intensity coefficient in each reference direction comprises: Determine the denoised noise signal in each reference direction based on the denoised signal strength coefficient in each reference direction; Performing decomposition and inverse transformation on the denoised noise signal in each reference direction to obtain a noise image in each reference direction; The noise images in each reference direction are fused to obtain the imaging features of the component.

7. The method according to claim 6, characterized in that The target image is subjected to signal decomposition processing based on a wavelet transform method; The denoised noise signal in each reference direction is decomposed and inversely transformed based on the inverse transformation method of the wavelet transformation method.

8. The method according to claim 1, characterized in that Extracting component imaging features of the target camera component in the target image includes: Obtaining an image cropping rule; the image cropping rule is used to define the size and position of uniformly cropping the image; Performing cropping processing on the target image according to the image cropping rule to obtain a cropped image of the target image; In the cropped image, the component imaging features are extracted.

9. The method according to claim 1, characterized in that: The target device has an associated device type; the method further comprising: Obtain the device classification network to be trained; Calling the device classification network to be trained to predict the device type of the target device based on the device fingerprint of the target device; Based on the difference between the device type predicted for the target device and the device type associated with the target device, the network parameters of the device classification network to be trained are modified to obtain a trained device classification network.

10. The method according to claim 9, characterized in that The method of correcting the network parameters of the device classification network to be trained based on the difference between the device type predicted for the target device and the device type associated with the target device to obtain a trained device classification network includes: Based on the device type predicted for the target device and the device type associated with the target device, generating a device classification loss of the device classification network for the target device; the device classification loss is used to reflect the difference between the device type predicted for the target device and the device type associated with the target device; The network parameters of the device classification network to be trained are modified based on the device classification loss to obtain the trained device classification network.

11. The method according to claim 9, characterized in that There are multiple target devices, one target device is associated with one device type, and multiple target devices are associated with multiple device types; One target device has one predicted device type, and the device type predicted for any target device among the multiple target devices is any one of the multiple device types.

12. The method according to claim 11, characterized in that The method further comprises: Acquire a verification image sent by a verification device; the verification image is obtained by real-time shooting of a camera component included in the verification device; Based on the verification image, generating a device fingerprint of the verification device; The trained device classification network is called to predict a target device type of the verification device among the multiple device types based on the device fingerprint of the verification device.

13. The method according to claim 12, characterized in that Each device type in the plurality of device types has an associated device service; The method further comprises: If the verification device sends a call request for a target device service before sending the verification image, and the target device service is consistent with the device service associated with the target device type, the target device service is provided to the verification device in response to the call request.

14. A device for processing equipment data, characterized in that: The device comprises: An acquisition module, used for acquiring a target image captured by a target camera component; the target camera component is a camera component in a target device; An extraction module, used to extract component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; A generating module is used to generate a device fingerprint of the target device based on the component imaging feature.

15. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

16. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 13.

Images