Method and apparatus for processing device data, and product, device and medium

Abstract

A method and apparatus for processing device data, and a product, a device and a medium. The method comprises: acquiring a target image, which is collected by a target camera assembly, wherein the target camera assembly is a camera assembly in a target device (S101); extracting an assembly imaging feature of the target camera assembly from the target image, wherein the assembly imaging feature is used for reflecting a non-uniform light response feature of the target camera assembly during imaging (S102); and generating a device fingerprint of the target device on the basis of the assembly imaging feature (S103). By means of the present application, the accuracy and stability of a device fingerprint, which is generated for a target device, can be improved.

Description

Device data processing method, apparatus, product, equipment and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 27, 2023, with application number 2023107702926 and application name “Method, device, product, equipment and medium for processing device data”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to device data processing technology. Background Art

[0003] The device fingerprint of a device is a device feature used to uniquely identify the device. In related technologies, the device fingerprint of a device can usually be generated based on some usage information of the device (such as the working information of the device's processor). However, since different devices may have the same or similar working conditions (such as the owners of different devices have the same or similar device usage habits), the device fingerprints generated for each device are not very different. In addition, since the working conditions of the device may change, the device fingerprint of the device may also change accordingly, making the device fingerprint of the device unstable and inaccurate. Therefore, how to generate a more accurate and stable device fingerprint for a device has become an urgent problem to be solved.

[0004] Summary of the Invention

[0005] This application provides a method, apparatus, product, device, and medium for processing device data, which can improve the accuracy and stability of device fingerprints generated for target devices.

[0006] On one hand, the present application provides a method for processing device data, which is executed by a computer device, and includes:

[0007] Acquire a target image captured by a target camera component; the target camera component is a camera component in a target device;

[0008] Extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response characteristics of the target camera component during imaging;

[0009] Generate device fingerprints of the target device based on component imaging features.

[0010] In one aspect, the present application provides a device data processing apparatus, the apparatus comprising:

[0011] An acquisition module, configured to acquire a target image captured by a target camera assembly; the target camera assembly is a camera assembly in a target device;

[0012] An extraction module is used to extract component imaging features of a target camera component in a target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging;

[0013] The generation module is used to generate a device fingerprint of the target device based on the component imaging features.

[0014] In one aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor executes the method in one aspect of the present application.

[0015] In one aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0016] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in various optional embodiments such as the above-mentioned aspect.

[0017] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and can extract component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and furthermore, can generate a device fingerprint of the target device based on the component imaging features. It can be seen that the method proposed in the present application can generate a device fingerprint of the target device based on the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, they have extremely high uniqueness and stability. Therefore, the device fingerprint of the target device having a strong binding association relationship with the target camera component is generated based on the component imaging features, which can ensure that the device fingerprint is very stable and can be used to accurately identify the target device having the target camera component. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1 is a schematic diagram of a network architecture provided by the present application;

[0019] FIG2 is a schematic diagram of a scenario for generating a device fingerprint provided by the present application;

[0020] FIG3 is a flow chart of a method for processing device data provided by the present application;

[0021] FIG4 is a schematic diagram of a variance estimation scenario provided by this application;

[0022] FIG5 is a flow chart of a feature extraction method provided by the present application;

[0023] FIG6 is a schematic diagram of a scenario of a network training method provided by the present application;

[0024] FIG7 is a schematic diagram of a network training scenario provided by the present application;

[0025] FIG8 is a schematic diagram of a process for classifying devices based on device fingerprints provided by the present application;

[0026] FIG9 is a schematic structural diagram of a device data processing apparatus provided by the present application;

[0027] FIG10 is a schematic structural diagram of a computer device provided in this application. DETAILED DESCRIPTION

[0028] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0029] This application primarily relates to machine learning in artificial intelligence. Specifically, machine learning techniques are used to train a device classification network to identify and classify devices based on their device fingerprints. For details, see the description of the embodiment corresponding to FIG6 below.

[0030] This application also involves blockchain-related technologies. Specifically, in this application, the device fingerprint generated for the device can be stored on the chain to ensure the security of the device fingerprint.

[0031] It should be noted that all data collected in this application (such as images collected by the target camera component, images collected by the verification device through the camera component, and other related data) are collected with the consent and authorization of the object to which the data belongs (such as the user, organization or enterprise), and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0032] Here, the relevant technical concepts involved in this application are explained:

[0033] Device fingerprint: A device characteristic used to uniquely identify a device, or a unique device identifier.

[0034] PRNU (Photo Response Non-Uniformity), also known as non-uniform photoresponse characteristics, is a type of noise characteristic. PRNU is the primary component of sensor pattern noise. Due to manufacturing process defects caused by variations in the thickness of the silicon coating between sensor pixels, slight variations in the photosensitivity of the imaging device's photosensitive elements occur. This results in a fixed error distribution across the entire photosensitive array of the imaging device's photosensitive element. This fixed error distribution is also implicitly reflected in images captured by the imaging device's photosensitive element as a multiplicative factor.

[0035] The photosensitive element of the imaging device may be a photosensitive element in a camera assembly (such as a camera in a device) (such as a light-sensitive sensor in a camera).

[0036] Please refer to Figure 1, which is a structural diagram of a network architecture provided by the present application. As shown in Figure 1, the network architecture may include a server 200 and a terminal device cluster. The terminal device cluster may include one or more terminal devices, and the number of terminal devices is not limited here. As shown in Figure 1, the multiple terminal devices may specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n; as shown in Figure 1, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all be connected to the server 200 through a network connection, so that each terminal device can exchange data with the server 200 through the network connection.

[0037] The server 200 shown in FIG1 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smart terminal such as a smartphone, tablet computer, laptop computer, desktop computer, smart TV, etc. The following describes the embodiments of the present application in detail using the communication between the terminal device 1 and the server 200 as an example.

[0038] Please refer to Figure 2 again, which is a schematic diagram of a scenario for generating a device fingerprint provided by this application. As shown in Figure 2, the terminal device 1 may include a camera component, which may be a camera in the terminal device 1. In this application, the terminal device 1 may be referred to as the target device, and the camera component in the target device may be referred to as the target camera component.

[0039] Optionally, the target camera assembly may include a lens, an anti-aliasing filter, a filter (CFA), and a sensor, etc., for imaging. The target camera assembly can shoot a scene, and the captured image passes through the lens, anti-aliasing filter, filter, and sensor in the target camera assembly in sequence, and finally a target image can be captured. The target image is the photo or video taken by the target camera assembly.

[0040] Here, since the sensor in the target camera assembly has pattern noise during the imaging process, the pattern noise may be PRNU noise. Therefore, the target image obtained by the above shooting will correspondingly contain the pattern noise.

[0041] The server 200 can obtain the target image (e.g., the target image can be captured using the target camera assembly and provided to the server 200 upon completion of production). The server 200 can then extract the imaging characteristics of the target camera assembly from the target image. This imaging characteristic can be the PRNU noise generated by the sensor in the target camera assembly. This imaging characteristic can be referred to as the component imaging characteristic, which can be used to reflect the non-uniform light response characteristics of the target camera assembly. The specific process of extracting the component imaging characteristic from the target image can be found in the description of the embodiment corresponding to FIG. 3 below.

[0042] Furthermore, the server 200 can generate a device fingerprint of the target device based on the above-mentioned component imaging features. Subsequently, the server 200 can accurately identify and verify the identity of the target device based on the generated device fingerprint of the target device. The details of this process can be found in the relevant descriptions of the following embodiments.

[0043] In the present application, since the component imaging feature of the target device is an inherent imaging feature of the target camera component in the target device, an accurate and stable device fingerprint can be generated for the target device based on the component imaging feature.

[0044] Please refer to Figure 3, which is a flow chart of a method for processing device data provided by the present application. The execution subject in the embodiment of the present application can be a device for processing device data (which can be called a data processing device), and the data processing device can be a computer device, or a computer device cluster composed of multiple computer devices. The computer device can be a server, a terminal device, or other devices, without limitation. As shown in Figure 3, the method may include:

[0045] Step S101 , obtaining a target image captured by a target camera component; the target camera component is a camera component in a target device.

[0046] The target device may include a camera component, which may be referred to as a target camera component. The target device may be a terminal device or other device, such as a merchant's cash register, a user's handheld terminal device, or a portable terminal device. The target camera component may be a component in the target device for capturing images, such as a camera in the target device having a sensor. Optionally, the target device may be an IoT device (Internet of Things device).

[0047] The data processing device can obtain a target image captured (e.g., photographed) by a target camera assembly. The target image can be any image captured by the target camera assembly. Alternatively, the target image can be a photo taken by the target device, or a video frame selected from a video captured by the target camera assembly. Subsequently, the data processing device can generate a device fingerprint of the target device based on the target image.

[0048] Optionally, the target image can be captured by the target camera assembly when the target camera assembly leaves the factory or when the target camera assembly is assembled into the target device, so as to pre-generate a device fingerprint of the target device before the target device is put into use, which is subsequently used to identify the target device.

[0049] Alternatively, the target image can also be acquired by calling the target camera component in the target device when the target device is put into use and the device fingerprint of the target device is needed to identify the target device. Here, calling the target camera component to capture the image is done after obtaining the consent of the user of the target device. For example, an inquiry pop-up window may pop up, and the user may click the option of agreeing in the inquiry pop-up window. After that, the target camera component may be called to capture the image or video, and the target device may send the captured image to the data processing device.

[0050] Step S102 : extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging.

[0051] The data processing device can extract the component imaging feature of the target camera component in the target image. The component imaging feature can be a PRNU feature extracted from the target image. The component imaging feature can be used to reflect the non-uniform light response characteristics of the target camera component during imaging. Since the PRNU feature is caused by the uneven thickness of the silicon coating at various positions of the photosensitive element of the target camera component, the PRNU feature is an inherent feature of the target camera component.

[0052] Specifically, the PRNU features contained in different images captured by the same camera assembly are identical. Even if there are differences in the PRNU features between different images captured by the same camera assembly, these differences are subtle and negligible. Therefore, the PRNU features contained in different images captured by the target camera assembly can be considered identical.

[0053] The following describes a specific process of extracting the component imaging features of the target camera component from the target image.

[0054] First, we introduce a formula to explain the PRNU feature. This formula is based on the noise extraction method of wavelet filtering to obtain PRNU noise (i.e., PRNU feature). This formula regards the image as a composite of a noise-free image and a noise image. As shown in the following formula: ij =f ij (x ij +η ij )+c ij +∈ ij (1)

[0055] Wherein, i represents the number of rows, j represents the number of columns, i is a positive integer less than or equal to the total number of rows of the silicon coating of the target camera component, and j is a positive integer less than or equal to the total number of columns of the silicon coating of the target camera component. ij The PRNU multiplicative noise factor used to determine the pixel position at the i-th row and j-th column of the silicon coating of the target camera component, f ij Can be a noise function related to PRNU noise. ij can represent the overall noise signal of the image defined by the above formula.

[0056] The size (which can be understood as the area) of the silicon coating of the target imaging component is equal to the size of the image captured by the target imaging component. Therefore, a pixel position of the silicon coating of the target imaging component corresponds to a pixel position of the target image. For example, the pixel position at the i-th row and j-th column of the silicon coating of the target imaging component corresponds to the pixel position at the i-th row and j-th column of the target image. Therefore, the PRNU multiplicative noise factor of the pixel position at the i-th row and j-th column of the silicon coating of the target imaging component can be the PRNU multiplicative noise factor of the pixel at the i-th row and j-th column of the target image.

[0057] x ij represents the incident light of the sensor (which may be a photosensitive element) of the target camera assembly, η ij represents shot noise, which is noise associated with PRNU and is caused by electron emission non-uniformity (emission non-uniformity caused by uneven thickness of silicon coating). This shot noise can also be called environmental noise.

[0058] c ij Represents dark current noise, which is the noise caused by the current when the target camera component takes an image. It is usually fixed and is a noise characteristic that the target camera component has when it leaves the factory. ∈ ij Indicates additional random noise. This is a relatively small amount of noise added to balance various noises. It can be a fixed value and can be present or absent. Dark current noise and additional random noise are not associated with PRNU noise.

[0059] Therefore, in this application, it is necessary to retain f ij (x ij +η ij ) and remove the noise of c ij +∈ ij This part of the noise.

[0060] Based on the principles described above, the specific process of extracting component imaging features of a target camera component from a target image in the present application can be described as follows.

[0061] First, the data processing device can perform signal decomposition processing on the target image to obtain a noise signal (which can be called an image noise signal) and a non-noise signal contained in the target image. The data processing device can use a wavelet transform method (such as a fourth-level wavelet transform (4-layer wavelet transform)) to perform signal decomposition processing on the target image. Optionally, the wavelet basis of the wavelet transform can default to db4 (a wavelet function that can be used to determine the waveform of the wavelet transform), and can also be other wavelet bases depending on the actual application scenario.

[0062] Since the above-mentioned dark current noise and additional random noise are usually noise of low-frequency signals, the present application can remove the low-frequency signal in the image noise signal obtained by the above decomposition to obtain a transition noise signal, that is, the noise signal obtained after removing the low-frequency signal in the image noise signal can be called a transition noise signal; in this way, the noise of the useless low-frequency signal is removed, which is conducive to determining more accurate and stable component imaging characteristics based on the transition noise signal obtained thereby.

[0063] The low-frequency signal may refer to a signal whose frequency is lower than a set frequency threshold, and the low-frequency signal is a signal that appears less frequently in the image noise signal. The frequency threshold may be set according to the actual application scenario.

[0064] The above-mentioned transition noise signal may include multiple parts of noise signals obtained by wavelet transform decomposition, and the multiple parts of noise signals may include noise signals of the target image in multiple directions (which may be called reference directions). Optionally, the multiple reference directions may include horizontal directions, vertical directions, diagonal directions, etc.

[0065] In other words, since the low-frequency signal (which can be called the low-frequency component) in the image noise signal has been removed, the transition noise signal can include a high-frequency noise component in the horizontal direction (which can also be called the high-frequency noise signal), a high-frequency noise component in the vertical direction, and a high-frequency noise component in the diagonal direction. It will be understood that the above-mentioned image noise signal can also include noise signals in the above-mentioned multiple reference directions (the noise signal in each reference direction can include a low-frequency noise signal and a high-frequency noise signal).

[0066] It should be noted that the target image may include images on multiple channels (such as the three channels R (red color channel), G (green color channel), and B (blue color channel)) (which can be understood as the image components of the target image on multiple channels). Therefore, the above-mentioned process of performing signal decomposition processing on the target image to obtain the image noise signal contained in the target image may include: performing signal decomposition processing (such as wavelet transform processing) on ​​the image components of the target image on each channel respectively to obtain the image noise signal of the target image on each channel. One channel may have one of the above-mentioned image noise signals.

[0067] Subsequently, the component imaging characteristics of the target camera assembly can be generated based on the image noise signals of the target image on each channel. Since the principles for processing the image components on each channel are the same and independent, the following description uses the image noise signal processing process on any one channel as an example.

[0068] In which, the noise signal in each reference direction contained in the above-mentioned transition noise signal can correspond to a signal strength coefficient, and the signal strength coefficient can be used to reflect the strength of the noise signal in the corresponding reference direction. The noise signal in a reference direction can correspond to a signal strength coefficient. The larger the signal strength coefficient, the greater the strength of the noise signal. Conversely, the smaller the signal strength coefficient, the smaller the strength of the noise signal.

[0069] The signal intensity coefficient corresponding to the noise signal in each reference direction in the transition noise signal may also be generated after performing signal decomposition processing on the target image. The signal intensity coefficient corresponding to the noise signal in each reference direction may be the wavelet coefficient in each reference direction obtained after performing signal decomposition processing on the target image. That is, the signal intensity coefficient in a reference direction may be the wavelet coefficient decomposed in the reference direction. The wavelet coefficient of the noise signal in a reference direction may be used to reflect the degree of similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform. The larger the wavelet coefficient, the greater the similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform. Conversely, the smaller the wavelet coefficient, the less similarity between the waveform of the noise signal in the reference direction and the waveform of the wavelet transform.

[0070] The data processing device can perform denoising on the signal strength coefficients corresponding to the noise signals in the multiple reference directions obtained above to obtain denoised signal strength coefficients in each reference direction. Furthermore, the data processing device can generate the component imaging feature using the denoised signal strength coefficients in the multiple reference directions contained in the transition noise signal.

[0071] Optionally, the process of the data processing device performing denoising on the signal strength coefficients corresponding to the noise signals in the multiple reference directions may include:

[0072] Among them, any reference direction among the above-mentioned multiple reference directions can be used as the target reference direction. The following is an example of denoising the signal strength coefficient corresponding to the noise signal in a reference direction of a transition noise signal on a channel (can be any one).

[0073] Optionally, the signal strength coefficient (i.e., wavelet coefficient) corresponding to the noise signal in the target reference direction can be a vector (which can be called a coefficient vector), which can contain multiple values ​​(i.e., multiple elements), and the multiple values ​​can be called multiple coefficient values ​​contained in the signal strength coefficient (i.e., the values ​​of the coefficients of the wavelet transform). In other words, the signal strength coefficient corresponding to the noise signal in the target reference direction can contain multiple coefficient values.

[0074] The data processing device can perform variance estimation (which can be local variance estimation) on multiple coefficient values ​​in the signal strength coefficient of the noise signal in the target reference direction (which can be referred to as the signal strength coefficient in the target reference direction) to obtain an estimated variance value in the target reference direction, wherein the local variance estimation can be achieved through an initial standard deviation, and the default value of the initial standard deviation can be 5 (or other values).

[0075] The process of estimating the local variance of the signal strength coefficient in the target reference direction may include:

[0076] The data processing device can obtain a target window (belonging to a sliding window), which is a window used for performing local variance estimation on the signal strength coefficient in the above-mentioned target reference direction. The target window has a set window size (which can be set according to the actual application scenario, such as 3, 5, 7 or 9, etc.). Furthermore, through the target window, multiple coefficient values ​​in the signal strength coefficient in the target reference direction can be slidingly estimated, and the sliding step size can also be determined according to the actual application scenario, such as 1 or 3, etc.

[0077] Each time the target window slides, a number of coefficient values ​​equal to the window size can be selected from the signal strength coefficients in the target reference direction. The target window can slide multiple times according to the step size, ultimately ensuring that all coefficient values ​​in the signal strength coefficients in the target reference direction are selected. Each time the target window slides, the data processing device can perform a variance estimation on the coefficient values ​​selected by the target window to obtain an estimated variance value of the coefficient values ​​selected by the target window. Each sliding position can correspond to an estimated variance value. Furthermore, the smallest estimated variance value among the estimated variance values ​​corresponding to the multiple sliding positions can be used as the estimated variance value in the target reference direction.

[0078] For example, if the signal strength coefficient in the target reference direction includes 9 coefficient values ​​(coefficient value 1 to coefficient value 9 in sequence), the size of the target window is 3, and the step size is 3, then when the target window is at the starting position, the coefficient values ​​1 to 3 of the 9 coefficient values ​​can be selected through the target window, and the variance of the coefficient values ​​1 to 3 can be estimated to obtain an estimated variance value of 1. Then, the target window is slid to the right by 3 coefficient values ​​(because the step size is 3), so that the target window selects the coefficient value 4 of the 9 coefficient values. ~ coefficient value 6, and the variance of the coefficient values ​​4 to 6 can be estimated to obtain an estimated variance value 2. Further, the data processing device slides the target window to the right by 3 coefficient values, so that the target window selects coefficient values ​​7 to 9 among the 9 coefficient values, and the variance of the coefficient values ​​7 to 9 can be estimated to obtain an estimated variance value 3. Finally, the smallest estimated variance value among the estimated variance values ​​1, 2 and 3 obtained by the above estimation can be used as the estimated variance value in the target reference direction.

[0079] Please refer to Figure 4, which is a schematic diagram of a variance estimation scenario provided by this application. As shown in Figure 4, the signal strength coefficient in the target reference direction can have 9 coefficient values, ranging from 1 to 9. The sliding window here can be the above-mentioned target window, where the step size of the target window can be 2 and the window size can be 3.

[0080] When the sliding window is at the starting position, coefficient values ​​1 to 3 of the signal strength coefficient in the target reference direction can be selected, and variance estimation can be performed on these coefficient values ​​1 to 3 to obtain an estimated variance value of 1. Furthermore, the sliding window is slid rightward by two coefficient values, and coefficient values ​​3 to 5 of the signal strength coefficient in the target reference direction can be selected, and variance estimation can be performed on these coefficient values ​​3 to 5 to obtain an estimated variance value of 2. Next, the sliding window is slid rightward by another two coefficient values ​​again, and coefficient values ​​5 to 7 of the signal strength coefficient in the target reference direction can be selected, and variance estimation can be performed on these coefficient values ​​5 to 7 to obtain an estimated variance value of 3. Furthermore, the sliding window is slid rightward by another two coefficient values ​​again, and coefficient values ​​7 to 9 of the signal strength coefficient in the target reference direction can be selected, and variance estimation can be performed on these coefficient values ​​7 to 9 to obtain an estimated variance value of 4. At this point, variance estimation has been performed on each coefficient value included in the signal strength coefficient in the target reference direction.

[0081] The data processing device may use the minimum value of the sum of the estimated variance value 1, the estimated variance value 2, the estimated variance value 3, and the estimated variance value 4 obtained above as the estimated variance value in the target reference direction.

[0082] After obtaining the estimated variance value in the target reference direction, denoising processing can be performed using the signal strength coefficient of the noise signal in the target reference direction to obtain the denoised signal strength coefficient in the target reference direction. Optionally, the process may include: inputting the signal strength coefficient in the target reference direction and the estimated variance value into a filter (such as a Wiener filter) for coefficient denoising, and then, the filter denoising the signal strength coefficient in the target reference direction based on the estimated variance value to obtain the denoised signal strength coefficient in the target reference direction.

[0083] It can be understood that the signal strength coefficient in the target reference direction can be filtered (i.e., denoised) by the Wiener filter to obtain the denoised signal strength coefficient, and the variance value of the denoised signal strength coefficient can be made close to the input estimated variance value to filter out the interference signal in the signal strength coefficient in the target reference direction and obtain a signal strength coefficient that tends to be stable.

[0084] The data processing device can perform denoising on the signal strength coefficients of the noise signals in each reference direction in the transition noise signals of each channel through the implementation method described above, and obtain the denoised signal strength coefficients corresponding to the noise signals in each reference direction in the transition noise signals of each channel.

[0085] By the above denoising process of the signal strength coefficient in each reference direction, the coefficient values ​​with relatively sudden changes (such as sudden high or sudden low) in the signal strength coefficient in each reference direction can be removed to obtain a more accurate and stable signal strength coefficient. It can be understood that by the denoising process of the signal strength coefficient in each reference direction, the noise x of the incident light can also be reduced. ij and shot noise η ij The noise signal with large mutation is removed, so that a more stable and stable component imaging feature can be obtained later (that is, the denoising process of the signal intensity coefficient can be understood as the denoising process for the initial component imaging feature).

[0086] Furthermore, the denoised signal strength coefficients in each reference direction (including the denoised signal strength coefficients in all channels) can be used to obtain the denoised noise signal in each reference direction. For example, the denoised signal strength coefficients can be combined with wavelets (a basic element of wavelet transform used to represent signals) to generate the denoised noise signal. The wavelets can be understood as musical notes in the audio scene, and the denoised signal strength coefficients can be understood as the pitch of the musical notes in the audio scene.

[0087] Optionally, the data processing device can perform a signal decomposition and inverse transformation (e.g., an inverse transformation method of a wavelet transform, i.e., an inverse wavelet transform) on the denoised noise signals in each of the reference directions, thereby generating a noise image in each reference direction. The noise images in multiple reference directions are then fused to obtain a final component imaging feature, which can be an image or a PRNU feature image. That is, the denoised noise signals in each reference direction can be decomposed and inversely transformed using an inverse wavelet transform.

[0088] Alternatively, optionally, the data processing device may first perform signal fusion (such as superposition) on the denoised noise signals in the above-mentioned respective reference directions to generate a fused denoised signal. Then, the data processing device may perform signal decomposition and inverse transformation (such as the inverse transformation method of the wavelet transform method, i.e., inverse wavelet transform) on the fused denoised signal to generate the final component imaging feature.

[0089] The component imaging features generated by the above two methods are the same. How to generate the component imaging features based on the denoised signals in the above multiple reference directions can be determined according to the actual application scenario.

[0090] Optionally, the denoised noise signal in each reference direction can be directly decomposed and inversely transformed in the above manner to obtain the noise image in each reference direction, and then the noise image in each reference direction can be fused and the noise image thus obtained can be used as the component imaging feature of the target device.

[0091] Alternatively, the present application can also decompose and inversely transform the denoised noise signal in each reference direction to obtain the noise image in each reference direction, and then use the noise image after fusion processing of the noise image in each reference direction as the initial component imaging feature of the target device. In order to further filter out the noise other than the PRNU feature in the initial component imaging feature, the present application can also perform noise filtering processing (such as zero-mean filtering processing) on ​​the initial component imaging feature (belonging to the image) to obtain a more accurate component imaging feature of the target device.

[0092] In this way, through the above method, the denoised noise signal in each reference direction is decomposed and inversely transformed (such as decomposition and inverse transformation based on the inverse transformation method of the wavelet transform method) to obtain the noise image in each reference direction, and then the noise images in each reference direction are fused to obtain the component imaging feature, which can ensure that the generated component imaging feature does not contain useless noise information, that is, it can ensure that the generated component imaging feature is accurate and stable and can uniquely characterize the target device.

[0093] Optionally, since the sizes of images captured by different camera components may be different, in order to adopt a unified method to generate component imaging features of various camera components, the present application can adopt a unified image cropping rule to uniformly crop the images captured by the camera components, so as to generate component imaging features of the camera components through the cropped images.

[0094] Therefore, the present application can also obtain the image cropping rule, and can use the image cropping rule to crop the above-mentioned target image (the images of the target image on the three channels of R channel, G channel and B channel can be cropped separately) to obtain a cropped image of the target image (which can include a cropped image of the target image on the R channel, a cropped image on the G channel and a cropped image on the B channel). Furthermore, the component imaging characteristics of the above-mentioned target camera component can be extracted from the cropped image. For example, the above-mentioned process of performing signal decomposition processing on the target image to obtain the image noise signal contained in the target image can be performing signal decomposition processing on the cropped image of the target image to obtain the image noise signal, that is, the image noise signal can be the noise signal of the cropped image of the target image.

[0095] Optionally, the image cropping rule can be used to define the size and position of cropping an image. The cropping size can be set according to the actual application scenario, such as the size of (224, 224) (the size of the cropped image of the target image can be (224, 224, 3), indicating that the sizes of the cropped images on the three channels are all (224, 224)). The cropping position can also be set according to the actual application scenario, such as the cropping position can be the center position of the target image. Therefore, the cropped image of the target image obtained by cropping can be an image with the above-mentioned cropping size at the center position of the target image.

[0096] In this way, before extracting the component imaging features, the target image is preprocessed, and a cropped image with a specific size is cropped from a specific position therein, and then the component imaging features are extracted based on the cropped image. This helps to standardize the component imaging feature extraction process and helps to ensure the accuracy and stability of the extracted component imaging features.

[0097] The component imaging features generated above can be used to reflect the noise pattern when the target device captures an image.

[0098] Please refer to Figure 5, which is a flow chart of a feature extraction method provided by this application. As shown in Figure 5, the process may include:

[0099] 1. The data processing device obtains an RGB image (an image with three color channels), which can be a cropped image of the target image.

[0100] 2. The data processing device performs a 4-level wavelet transform on the RGB image in a single channel (any of the three color channels). A 4-level wavelet transform is a multi-dimensional (here, four-dimensional) wavelet transform of the RGB image. Wavelet bases for wavelet transforms in different dimensions can differ. Therefore, after performing wavelet transforms on a signal in different dimensions, different signal strength coefficients can be decomposed. The principle of performing wavelet transforms on the signal in each dimension is the same and independent. By performing a multi-dimensional wavelet transform on a signal (such as the RGB image described above), the signal can be decomposed and characterized in multiple dimensions, subsequently obtaining more accurate PRNU features.

[0101] 3. Perform local variance estimation and Wiener filtering on the horizontal high-frequency component of a single level (a single layer refers to any layer in the four-layer wavelet transform) in the above four-layer wavelet transform (such as the noise signal in the horizontal direction of the above transition noise signal) to obtain the denoised signal strength coefficient in the horizontal direction of the single level.

[0102] 4. Perform local variance estimation and Wiener filtering on the vertical high-frequency component of the single level (such as the noise signal in the vertical direction of the transition noise signal) to obtain the denoised signal strength coefficient in the vertical direction of the single level.

[0103] 5. Perform local variance estimation and Wiener filtering on the diagonal high-frequency component of the single level (such as the noise signal in the diagonal direction of the transition noise signal) to obtain the denoised signal strength coefficient in the diagonal direction of the single level.

[0104] 6. Determine whether the noise signals (such as transition noise signals) on the above four levels have been processed, that is, whether the denoised signal strength coefficients in each reference direction of each level have been obtained. If so, perform step 7 below. If not, perform step 3 above again to denoise the signal strength coefficients corresponding to the noise signals in each reference direction on the next level (next layer).

[0105] 7. Perform inverse wavelet transform and zero-mean filtering on the denoised noise signal in each reference direction on the above single channel (such as any one of the above three color channels) to generate a noise image on the single channel.

[0106] 8. Determine whether the noise images on the above three color channels have been obtained. If the noise images on the above three color channels have been obtained, you can execute the following step 9. If the noise images on the above three color channels have not been obtained, you can execute the above step 2 and subsequent steps again, and then obtain the noise image on the next channel according to the above process.

[0107] 9. The final PRNU feature image (i.e., component imaging feature) can be generated through the noise images on the three color channels. For example, the noise images in multiple reference directions on each color channel can be fused separately to obtain the fused noise images on each color channel. The final component imaging feature can include the fused noise images on each color channel.

[0108] Step S103: Generate a device fingerprint of the target device based on the component imaging features.

[0109] Optionally, the data processing device may adopt any of the following methods to generate a device fingerprint of the target device by using the component imaging features of the target camera component extracted from the target image.

[0110] The data processing device can directly image the features of the component as the device fingerprint of the target device.

[0111] The data processing device may obtain local features from the component imaging features as the device fingerprint of the target device. For example, the data processing device may capture a local image from the component imaging features as the device fingerprint of the target device.

[0112] The data processing device can also integrate some device attribute information of the target device itself into the component imaging feature to generate a device fingerprint of the target device. For example, the device attribute information of the target device may include the hardware address (MAC address), device identification, or device type of the target device, etc. Based on the device attribute information of the target device, an attribute image of the target device can be generated. For example, the attribute image can contain text of the attribute information of the target device, and the size of the attribute image can be the same as the size of the component imaging feature. Then, the data processing device can superimpose the attribute image and the component imaging feature of the target camera component (such as adding the pixel values ​​at the same pixel position) to generate the device fingerprint of the target device.

[0113] Through the above-mentioned method of the present application, a unique and stable device fingerprint of the target device can be generated through the imaging characteristics of the target camera component in the target device (such as the PRNU characteristics). The device fingerprint can subsequently be applied in any scenario where the target device needs to be identified, such as in the scenario where the target device is identified and verified through the device fingerprint.

[0114] The device fingerprint generation method of the present application has the following advantages: First, since the device captures images very conveniently, the computational complexity of extracting the PRNU features from the target image is also small, and therefore, the device fingerprint of the device can be generated more conveniently; second, since the PRNU features of various camera components are inherent and unique, the device fingerprint generated based on the PRNU features can achieve a higher degree of differentiation accuracy between different camera components; third, since the PRNU features are strongly associated with the photosensitive elements of the camera components, the device fingerprint generated based on the PRNU features is also more stable.

[0115] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and can extract component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and furthermore, can generate a device fingerprint of the target device based on the component imaging features. It can be seen that the method proposed in the present application can generate a device fingerprint of the target device based on the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, they have extremely high uniqueness and stability. Therefore, the device fingerprint of the target device having a strong binding association relationship with the target camera component is generated based on the component imaging features, which can ensure that the device fingerprint is very stable and can be used to accurately identify the target device having the target camera component.

[0116] Please refer to Figure 6, which is a schematic diagram of a scenario of a network training method provided by this application. As shown in Figure 6, the method may include:

[0117] Step S201: Obtain a device classification network to be trained.

[0118] Optionally, the data processing device may obtain a device classification network to be trained, where the device classification network to be trained may be a neural network model for classifying devices based on device fingerprints of the devices.

[0119] Optionally, the device classification network in the present application can be SOTA (target recognition) EfficientNet-B4 (an efficient convolutional neural network). Of course, it can also be other network structures, and the embodiments of the present application do not impose any restrictions on this.

[0120] Step S202 : calling a device classification network to determine a predicted device type corresponding to the target device based on the device fingerprint of the target device.

[0121] Optionally, there are multiple target devices, and one target device can have an associated device type. The device type associated with the target device refers to the actual device type of the target device (i.e., the real device type). In the process of training the device classification network, the device type associated with the target device can be used as a label for model training. Multiple target devices can be associated with a total of multiple device types. The target devices of these multiple device types can be used to perform classification training on the device classification network to be trained, so that the device classification network to be trained can subsequently identify the multiple device types. It should be understood that multiple target devices and multiple device types may not have a one-to-one correspondence, that is, there may be at least two target devices corresponding to the same device type.

[0122] Optionally, in order to enable the device classification network to be trained to learn the features of the multiple device types more balanced and accurately, the number of target devices of the above-mentioned various device types can be relatively uniform, such as the number of target devices of each device type can be the same, or the absolute value of the difference between the number of target devices of any two device types is less than or equal to the target value, and the target value can be a smaller value (can be a positive integer).

[0123] The data processing device can call the device classification network to be trained, and predict the device type of each target device based on the device fingerprint of each target device (the device fingerprint obtained by the above-mentioned component imaging features), that is, obtain the predicted device type corresponding to each target device. One target device can correspond to one predicted device type. For any target device, its corresponding predicted device type is any one of the multiple device types of the above-mentioned multiple target devices.

[0124] Optionally, the data processing device calls the device classification network to be trained, and the predicted device type corresponding to the determined target device can be represented by a vector (which can be a probability distribution, which can be called a predicted classification probability). The vector can include the probability that the device type of the predicted target device is each of the above-mentioned multiple device types. The predicted device type corresponding to the target device can be the device type corresponding to the largest probability in the vector.

[0125] Step S203 : adjusting network parameters of the device classification network based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device.

[0126] The data processing device may modify the network parameters of the device classification network to be trained based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device. This process may include:

[0127] The data processing device may generate a device classification loss for the target device (which may include the sum of the device classification losses for each target device) of the device classification network based on the predicted device type corresponding to the target device and the device type associated with the target device. The device classification loss may be used to indicate the difference between the predicted device type corresponding to the target device and the device type associated with the target device. A larger device classification loss indicates a larger difference between the predicted device type corresponding to the target device and the device type associated with the target device. Conversely, a smaller device classification loss indicates a smaller difference between the predicted device type corresponding to the target device and the device type associated with the target device.

[0128] For example, the device type associated with the target device can also be represented by a vector (which can be called the true classification probability), in which only the probability value corresponding to the device type associated with the target device is 1, and the probability values ​​corresponding to other device types are 0. Therefore, optionally, the classification loss for a target device can be the cross-entropy loss between the true classification probability and the predicted classification probability of the target device for the above-mentioned multiple device types.

[0129] The data processing device can use the device classification loss obtained for the target device to modify the network parameters of the device classification network to be trained. Once the network parameters of the device classification network to be trained are modified, a trained device classification network can be obtained. The goal of modifying the network parameters of the device classification network to be trained is to minimize the device classification loss (e.g., to zero).

[0130] Optionally, the completion of network parameter correction of the device classification network to be trained may refer to correcting the network parameters of the device classification network to be trained to a convergence state, or refers to the number of iterative training of the network parameters of the device classification network to be trained according to the above principles reaching (greater than or equal to) a threshold number.

[0131] Please refer to Figure 7, which is a schematic diagram of a network training scenario provided by this application. As shown in Figure 7, a data processing device can input the device fingerprint of a target device into a device classification network to be trained. The device classification network to be trained can determine the predicted device type corresponding to the target device based on the input device fingerprint.

[0132] Furthermore, the data processing device can generate a classification loss of the device classification network to be trained for the target device (i.e., the above-mentioned device classification loss) based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device. Furthermore, the data processing device can backpropagate the device classification loss to the device classification network to be trained, and correct the network parameters of the device classification network to be trained through the device classification loss to obtain a trained device classification network.

[0133] The trained device classification network can then be used to classify devices by device type based on their fingerprints. Optionally, the camera components in target devices of the same device type are typically produced in the same batch using the same method. The camera components generated in this batch typically have the same imaging characteristics. For example, the camera components generated in this batch may be camera components in the same batch of cash registers.

[0134] In this way, through the above method, training a device classification network for identifying device types based on device fingerprints can expand the application scenarios of the device fingerprints determined by the method provided in the embodiments of the present application, so that the device fingerprint can be used in device classification scenarios. Through the trained device classification network, the type of the device can be accurately determined based on the device fingerprint generated by the component imaging features, thereby improving the classification accuracy of the device.

[0135] The following example describes the process of verifying and identifying a device using the trained device classification network.

[0136] The data processing device can obtain a verification image sent by a verification device, which can be a front-end device, such as a cash register. The verification image can be an image captured by the camera component of the verification device. For example, the verification image can be captured when the verification device requests the data processing device to invoke a target service (which can be any service, specifically determined based on the actual application scenario, such as a device service for the entire verification device or a service within an application within the verification device). The verification image can also be captured by the data processing device after the verification device requests the data processing device to invoke the target service. The verification image can also be captured by the verification device and sent to the data processing device proactively.

[0137] For example, if the verification device is a cash register, the verification image may be an image captured in real time by the verification device after the cash register initiates a request to the data processing device to invoke a payment collection service, or while the cash register initiates a request to the data processing device to invoke a payment collection service. The verification device may include a program capable of capturing images in real time, and the data processing device may also include an interface for receiving images captured in real time. Thus, the data processing device can securely and accurately receive the verification image captured in real time by the verification device through this interface, thereby ensuring that the received verification image was captured in real time by the verification device.

[0138] The data processing device generates a device fingerprint of the verification device based on the received verification image and in accordance with the principle of generating the device fingerprint of the target device.

[0139] Furthermore, the data processing device can call the above-mentioned trained device classification network, and determine the target device type corresponding to the verification device among multiple device types based on the device fingerprint of the verification device. The device type with the highest predicted probability for the verification device and greater than the probability threshold among the multiple device types can be used as the target device type predicted for the verification device.

[0140] The above-mentioned multiple device types have their own associated device services. A device service associated with a device type may refer to a device service that can be called by a device of the device type, that is, a device may have the authority to call the device service associated with the device type to which it belongs.

[0141] Therefore, if the verification device initiates a call request for the target device service, for example, the verification device sends a call request for the target device service before sending the above-mentioned verification image, or sends a call request for the target device service when sending the above-mentioned verification image, and the requested target device service is consistent with the device service associated with the target device type to which the verification device belongs (i.e., the same), then the data processing device can provide the target device service to the verification device in response to the call request, such as providing payment collection services to the verification device.

[0142] It can be understood that even in some special circumstances, such as when the device fingerprint of the verification device is leaked, the method provided by the present application can also be used to securely and accurately identify the device type (device identity) of the device through the real-time captured image, because the data processing device does not directly receive the device fingerprint, that is, it does not directly verify the received device fingerprint, but receives the image captured by the verification device in real time, and then verifies the device fingerprint generated based on the image. Even if the device fingerprint of the verification device is leaked in advance, the PRNU feature unique to the verification device cannot be added to other images and then given to the data processing device. Verifying the real-time captured image further isolates the possibility of forging a local image containing the PRNU feature unique to the target device and giving it to the data processing device to skip the identity identification of the target device. Therefore, by using the method provided by the present application, the identity of the device containing the target camera component can be securely and accurately identified based on the component imaging features of the target camera component.

[0143] By adopting the above-mentioned method of the present application, the device classification network can be called to generate a device fingerprint based on the component imaging characteristics of the camera component in the device to accurately and automatically identify the identity (i.e., type) of the device. Then, subsequent business processing can be performed on the verification device based on the identification result (such as related processing of device services associated with the device type of the verification device). In this way, according to the device type identification result of the verification device, the corresponding target device service is provided to the verification device, which can ensure that the corresponding device is provided with the device service it can use, thereby improving the security of the device service.

[0144] Please refer to Figure 8, which is a schematic diagram of a process for classifying devices based on device fingerprints provided by this application. Figure 8 corresponds to an embodiment that describes the overall solution process of this application. As shown in Figure 8, the process may include:

[0145] S1: The data processing device may collect device videos, which may be videos shot (ie, captured) by the target camera component contained in the target device.

[0146] S2: The data processing device can sample and extract the device video, that is, sample (can be random sampling, or can be uniform sampling (such as sampling one video frame every same number of video frames)) multiple video frames from the device video for subsequent network training. The sampled video frames can be used as the above-mentioned target images.

[0147] S3: The data processing device may perform image cropping processing on the sampled target image, such as performing cropping processing according to the above-mentioned image cropping rules, to obtain a cropped image of the target image.

[0148] S4: The data processing device may extract the PRNU feature (ie, component imaging feature) of the target camera component of the target device from the cropped image of the target image, and may generate a device fingerprint of the target device based on the PRNU feature.

[0149] Step S4 may include steps S41 to S44: In step S41, the data processing device may perform a multi-scale (such as the aforementioned 4-level) wavelet transform on the cropped image of the target image. In steps S42 and S43, the data processing device may perform local variance estimation and Wiener filtering on the multi-scale wavelet transform results to obtain a denoised noise signal of the cropped image. In step S44, the data processing device may perform an inverse wavelet transform on the denoised noise signal to obtain the PRNU characteristics of the target camera component in the target device.

[0150] S5: The data processing device may perform neural network training based on the device fingerprint of the target device generated above, such as training the device classification network to be trained to obtain a trained device classification network.

[0151] S6: The data processing device can perform network prediction through the trained device classification network, such as predicting the device type of the verification device based on the device fingerprint of the verification device through the trained device classification network.

[0152] S7: The data processing device can identify the verification device by using the device type predicted by the above-mentioned trained device classification network, such as identifying whether the verification device has the calling authority for the currently initiated target service, and then obtain an identification result for the verification. If the identification result is used to indicate that the verification device has the calling authority for the currently initiated target service, the data processing device can provide the target service to the verification device.

[0153] Please refer to Figure 9, which is a schematic diagram of the structure of a device data processing apparatus provided by this application. As shown in Figure 9, the device data processing apparatus 1 may include: an acquisition module 11, an extraction module 12 and a generation module 13.

[0154] An acquisition module 11 is configured to acquire a target image captured by a target camera assembly; the target camera assembly is a camera assembly in a target device;

[0155] The extraction module 12 is used to extract the component imaging characteristics of the target camera component in the target image; the component imaging characteristics are used to reflect the non-uniform light response characteristics of the target camera component during imaging;

[0156] The generating module 13 is configured to generate a device fingerprint of the target device based on the component imaging features.

[0157] Optionally, the extraction module 12 extracts component imaging features of the target camera component in the target image by:

[0158] Perform signal decomposition processing on the target image to obtain the image noise signal contained in the target image;

[0159] Based on the image noise signal, a component imaging feature of the target camera component is generated.

[0160] Optionally, the extraction module 12 generates the component imaging feature of the target camera component based on the image noise signal, including:

[0161] Remove the low-frequency signal from the image noise signal to obtain a transition noise signal;

[0162] Based on the transition noise signal, a component imaging signature is generated.

[0163] Optionally, the transition noise signal includes noise signals of the target image in multiple reference directions, and the noise signal in each reference direction corresponds to a signal strength coefficient, which is obtained by performing signal decomposition processing on the target image;

[0164] The extraction module 12 generates component imaging features based on the transition noise signal, including:

[0165] Denoising the signal strength coefficient corresponding to the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction;

[0166] Based on the denoised signal intensity coefficient in each reference direction, component imaging features are generated.

[0167] Optionally, the signal strength coefficient corresponding to the noise signal in the reference direction includes multiple coefficient values;

[0168] The extraction module 12 performs denoising on the signal strength coefficient corresponding to the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction, including:

[0169] performing variance estimation processing on a plurality of coefficient values ​​in a signal strength coefficient corresponding to a noise signal in a target reference direction to obtain an estimated variance value in the target reference direction; the target reference direction is any one of the plurality of reference directions;

[0170] Based on the estimated variance value, the signal strength coefficient corresponding to the noise signal in the target reference direction is denoised to obtain the denoised signal strength coefficient in the target reference direction.

[0171] Optionally, the extraction module 12 generates the component imaging feature based on the denoised signal intensity coefficient in each reference direction, including:

[0172] Determining a denoised noise signal in each reference direction based on a denoised signal strength coefficient in each reference direction;

[0173] The denoised noise signal in each reference direction is decomposed and inversely transformed to obtain the noise image in each reference direction;

[0174] The noise images in each reference direction are fused to obtain the component imaging features.

[0175] Optionally, the target image is subjected to signal decomposition processing based on a wavelet transform method;

[0176] The extraction module 12 decomposes and inversely transforms the denoised noise signal in each reference direction to obtain a noise image in each reference direction, including:

[0177] Based on the inverse transformation method of the wavelet transform method, the denoised noise signal in each reference direction is decomposed and inversely transformed to obtain the noise image in each reference direction.

[0178] Optionally, the extraction module 12 extracts component imaging features of the target camera component in the target image by:

[0179] Get the image cropping rule; the image cropping rule is used to define the size and position of the image cropping;

[0180] The target image is cropped according to the image cropping rule to obtain a cropped image of the target image;

[0181] In the cropped image, component imaging features are extracted.

[0182] Optionally, the target device has an associated device type; the apparatus 1 is further configured to:

[0183] Obtain the device classification network to be trained;

[0184] Call the device classification network to be trained and determine the predicted device type corresponding to the target device based on the device fingerprint of the target device;

[0185] Based on a difference between a predicted device type corresponding to the target device and a device type associated with the target device, network parameters of the device classification network are adjusted.

[0186] Optionally, the apparatus 1 adjusts the network parameters of the device classification network based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device, including:

[0187] Based on the predicted device type corresponding to the target device and the device type associated with the target device, generating a device classification loss of the device classification network for the target device; the device classification loss is used to reflect the difference between the predicted device type corresponding to the target device and the device type associated with the target device;

[0188] Adjust the network parameters of the device classification network based on the device classification loss.

[0189] Optionally, there are multiple target devices, one target device is associated with one device type, and multiple target devices are associated with multiple device types;

[0190] One target device corresponds to one predicted device type, and the predicted device type is any one of multiple device types.

[0191] Optionally, the device 1 is further used for:

[0192] Obtaining a verification image sent by the verification device; the verification image is captured by the verification device using a camera component;

[0193] generating a device fingerprint of the verification device based on the verification image;

[0194] The device classification network is called to predict the target device type to which the verification device belongs among multiple device types based on the device fingerprint of the verification device.

[0195] Optionally, multiple device types have their own associated device services;

[0196] The above device 1 is also used for:

[0197] If the verification device initiates a call request for the target device service, and the target device service is consistent with the device service associated with the target device type, the target device service is provided to the verification device in response to the call request.

[0198] According to one embodiment of the present application, the steps involved in the device data processing method shown in FIG3 can be performed by various modules in the device data processing apparatus 1 shown in FIG9 . For example, step S101 shown in FIG3 can be performed by the acquisition module 11 in FIG9 , step S102 shown in FIG3 can be performed by the extraction module 12 in FIG9 , and step S103 shown in FIG3 can be performed by the generation module 13 in FIG9 .

[0199] The present application can obtain a target image captured by a target camera component; the target camera component is a camera component in a target device; and can extract component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; and furthermore, can generate a device fingerprint of the target device based on the component imaging features. It can be seen that the device proposed in the present application can generate a device fingerprint of the target device based on the component imaging features of the target camera component in the target device. Since the component imaging features are used to reflect the non-uniform light response features that are exclusive and inherent to the target camera component, they have extremely high uniqueness and stability. Therefore, the device fingerprint of the target device having a strong binding association relationship with the target camera component is generated based on the component imaging features, which can ensure that the device fingerprint is very stable and can be used to accurately identify the target device having the target camera component.

[0200] According to one embodiment of the present application, the various modules in the device data processing device 1 shown in Figure 9 can be individually or all combined into one or several units to constitute, or one (some) of the units can be further divided into multiple functionally smaller sub-units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In actual applications, the functions of a module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the device data processing device 1 may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0201] According to one embodiment of the present application, a computer program capable of executing the steps involved in the corresponding methods shown in the various embodiments of the present application can be run on a general-purpose computer device (the computer device may include processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM)) to construct a device data processing apparatus 1 as shown in Figure 9. The computer program can be recorded on a computer-readable recording medium, for example, and can be loaded into the computer device via the computer-readable recording medium and run therein.

[0202] Please refer to Figure 10, which is a schematic diagram of the structure of a computer device provided in this application. As shown in Figure 10, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and optionally the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 10, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0203] In the computer device 1000 shown in Figure 10, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to implement the device data processing method provided in the embodiment of the present application.

[0204] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the device data processing methods described in the various embodiments of the present application, and can also execute the device data processing apparatus 1 described in the embodiment corresponding to FIG9 above, and thus will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here.

[0205] In addition, it should be noted that the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the processor executes the computer program, it can perform the description of the device data processing method in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0206] As an example, the above-mentioned computer program can be deployed and executed on one computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network. Multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain network.

[0207] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped with the computer device. Furthermore, the computer-readable storage medium may include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is about to be output.

[0208] The present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the description of the method for processing the above-mentioned device data in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0209] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0210] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0211] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for processing device data, executed by a computer device, the method comprising: Acquire a target image captured by a target camera assembly; The target camera component is a camera component in a target device; Extracting component imaging features of the target camera component in the target image; the component imaging features are used to reflect the non-uniform light response features of the target camera component during imaging; Based on the component imaging features, a device fingerprint of the target device is generated.

2. The method according to claim 1, wherein extracting component imaging features of the target camera component in the target image comprises: Performing signal decomposition processing on the target image to obtain an image noise signal contained in the target image; Based on the image noise signal, the component imaging feature of the target imaging component is generated.

3. The method according to claim 2, wherein generating the component imaging feature of the target camera component based on the image noise signal comprises: Removing low-frequency signals from the image noise signal to obtain a transition noise signal; Based on the transition noise signal, the component imaging feature is generated.

4. The method according to claim 3, wherein the transition noise signal comprises noise signals of the target image in multiple reference directions, the noise signal in one reference direction corresponds to a signal strength coefficient, and the signal strength coefficient is obtained after performing signal decomposition processing on the target image; The step of generating the component imaging feature based on the transition noise signal comprises: Performing denoising processing on the signal strength coefficient corresponding to the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction; The component imaging feature is generated based on the denoised signal intensity coefficient in each of the reference directions.

5. The method according to claim 4, wherein the signal strength coefficient corresponding to the noise signal in the reference direction comprises a plurality of coefficient values; The denoising process is performed on the signal strength coefficient corresponding to the noise signal in each reference direction in the transition noise signal to obtain the denoised signal strength coefficient in each reference direction, including: Performing variance estimation processing on a plurality of coefficient values ​​in a signal strength coefficient corresponding to a noise signal in a target reference direction to obtain an estimated variance value in the target reference direction; The target reference direction is any one of the multiple reference directions; Based on the estimated variance value, a signal strength coefficient corresponding to the noise signal in the target reference direction is denoised to obtain a denoised signal strength coefficient in the target reference direction.

6. The method according to claim 4 or 5, wherein generating the component imaging feature based on the denoised signal intensity coefficient in each reference direction comprises: Determine the denoised noise signal in each reference direction based on the denoised signal strength coefficient in each reference direction; Decomposing and inversely transforming the denoised noise signal in each reference direction to obtain a noise image in each reference direction; The noise image in each reference direction is fused to obtain the imaging feature of the component.

7. The method according to claim 6, wherein the target image is subjected to signal decomposition processing based on a wavelet transform method; and the decomposition and inverse transformation of the denoised noise signal in each reference direction is performed to obtain a noise image in each reference direction, comprising: Based on the inverse transformation method of the wavelet transformation method, the denoised noise signal in each reference direction is decomposed and inversely transformed to obtain a noise image in each reference direction.

8. The method according to any one of claims 1 to 7, wherein extracting component imaging features of the target camera component in the target image comprises: Obtaining an image cropping rule; the image cropping rule is used to define the size and position of cropping the image; Performing cropping processing on the target image according to the image cropping rule to obtain a cropped image of the target image; In the cropped image, the component imaging features are extracted.

9. The method according to any one of claims 1 to 8, wherein the target device has an associated device type; the method further comprising: Obtain the device classification network to be trained; Calling the device classification network to determine a predicted device type corresponding to the target device based on the device fingerprint of the target device; Based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device, a network parameter of the device classification network is adjusted.

10. The method according to claim 9, wherein adjusting the network parameters of the device classification network based on the difference between the predicted device type corresponding to the target device and the device type associated with the target device comprises: generating a device classification loss of the device classification network for the target device based on the predicted device type corresponding to the target device and the device type associated with the target device; The device classification loss is used to reflect the difference between the predicted device type corresponding to the target device and the device type associated with the target device; A network parameter of the device classification network is adjusted based on the device classification loss.

11. The method according to claim 9 or 10, wherein there are multiple target devices, one target device has one associated device type, and multiple target devices have multiple associated device types; in, One target device corresponds to one predicted device type, and the predicted device type is any one of the multiple device types.

12. The method according to claim 11, further comprising: Obtaining a verification image sent by a verification device; The verification image is obtained by the verification device calling the camera component therein to take pictures; Based on the verification image, generating a device fingerprint of the verification device; The device classification network is called to predict the target device type to which the verification device belongs from among the multiple device types based on the device fingerprint of the verification device.

13. The method of claim 12, wherein the plurality of device types have respective associated device services; The method further comprises: If the verification device initiates a call request for a target device service, and the target device service is consistent with a device service associated with the target device type, then in response to the call request, the target device service is provided to the verification device. Preparation service.

14. A device for processing device data, the device comprising: An acquisition module, used to acquire a target image captured by a target camera assembly; The target camera component is a camera component in a target device; An extraction module, used for extracting component imaging features of the target camera component in the target image; The component imaging feature is used to reflect the non-uniform light response feature of the target camera component during imaging; A generating module is used to generate a device fingerprint of the target device based on the component imaging feature.

15. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 13.

17. A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 13.

Images