A method and apparatus for identifying injection attack behavior

Abstract

The embodiment of the specification provides a method and device for identifying injection attack behavior, the method comprises the following steps: obtaining first data corresponding to an image acquisition process of a mobile terminal for a first object, wherein the first data comprises sensor data, the sensor data corresponds to a sensor in the mobile terminal, and the sensor comprises a first sensor for sensing the motion of the mobile terminal; obtaining image data corresponding to the image acquisition process, wherein the image data is identified from a first image corresponding to the image acquisition process, and the image data is used for indicating the motion of the mobile terminal; and determining whether the image acquisition process involves injection attack behavior by using the first data and the image data, so as to provide a new method for identifying injection attack behavior.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a method and apparatus for identifying injection attack behavior. Background Technology

[0002] eKYC (Electronic Know Your Customer) is a digital identity verification solution based on artificial intelligence technology. Institutions in the fields of financial services, banking, insurance, telecommunications, and e-commerce often use eKYC systems to verify the identity of their users.

[0003] Currently, eKYC systems typically authenticate users by using the mobile terminal's camera to capture images of the user's face and / or identification documents. These images (or videos, i.e., image sequences) are then used for user authentication. However, with the continuous development of AIGC (Artificial Intelligence Generated Content) technology, malicious actors have been exploiting AIGC to generate images (or videos) of faces and / or identification documents to launch injection attacks against eKYC systems. In these attacks, malicious actors exploit vulnerabilities in mobile terminals to inject generated images (or videos) of faces and / or identification documents, replacing the actual images (or videos) captured by the mobile terminal's camera, into the eKYC system's image (or video) verification unit.

[0004] Currently, in authentication systems such as eKYC, defense against the aforementioned injection attacks relies solely on the image (or video) itself. However, due to the continuous development of AIGC technology, the forged images (or videos) are becoming increasingly realistic, making defense based solely on the image (or video) itself increasingly difficult. To ensure the security of customer privacy information and / or assets for various organizations, it is crucial to provide a new method for identifying and defending against injection attacks. Summary of the Invention

[0005] This specification provides one or more embodiments of a method and apparatus for identifying injection attack behavior, so as to achieve the identification of injection attack behavior.

[0006] According to the first aspect, a method for identifying injection attack behavior is provided, including:

[0007] Acquire first data corresponding to the image acquisition process of the mobile terminal for the first object, wherein the first data includes sensor data, the sensor data corresponds to the sensors in the mobile terminal, and the sensors include a first sensor for sensing the motion of the mobile terminal;

[0008] Image data corresponding to the image acquisition process is acquired, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal;

[0009] Using the first data and the image data, determine whether the image acquisition process involves injection attacks.

[0010] According to the second aspect, a device for identifying injection attack behavior is provided, comprising:

[0011] The first acquisition module is configured to acquire first data corresponding to the image acquisition process of the mobile terminal for the first object, wherein the first data includes sensor data, the sensor data corresponds to the sensors in the mobile terminal, and the sensors include a first sensor for sensing the motion of the mobile terminal;

[0012] The second acquisition module is configured to acquire image data corresponding to the image acquisition process, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal;

[0013] The first determining module is configured to use the first data and the image data to determine whether the image acquisition process involves injection attack behavior.

[0014] According to a third aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0015] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect.

[0016] According to the injection attack identification method and apparatus provided in the embodiments of this specification, first data corresponding to the image acquisition process of a mobile terminal targeting a first object is acquired, wherein the first data includes sensor data, the sensor data corresponds to the sensors in the mobile terminal, and the sensors include a first sensor for sensing the movement of the mobile terminal; image data corresponding to the image acquisition process is acquired, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal; using the first data and the image data, it is determined whether the image acquisition process involves injection attack behavior.

[0017] The above process provides a novel method for identifying injection attacks. Considering that in a real image acquisition process (i.e., an image acquisition process without injection attacks), the movement of the mobile terminal during image acquisition is correlated with the movement of the first object in the acquired image sequence. For example, if the mobile terminal moves to the right relative to the first object, the corresponding image sequence will show that the first object (and / or its background) moves to the left relative to the mobile terminal. Similarly, if the mobile terminal moves closer to the first object, the corresponding image sequence will show that the first object (and / or its background) becomes larger. In other words, the image sequence can also indicate the movement of the mobile terminal. Therefore, this method identifies whether the image acquisition process involves injection attacks by identifying the correlation between the first data (sensor data, which can characterize the movement of the mobile terminal during image acquisition) and the image data (which can also indicate the movement of the mobile terminal). For example, it determines whether the movement of the mobile terminal indicated by the two is consistent. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 This is a schematic diagram illustrating the implementation framework of one embodiment disclosed in this specification;

[0020] Figure 2 A flowchart illustrating a method for identifying injection attack behavior provided in an embodiment;

[0021] Figure 3 This is a schematic diagram of the identification process for injection attack behavior provided in an embodiment;

[0022] Figure 4 A schematic diagram illustrating the process of determining image data provided in this embodiment;

[0023] Figure 5 A schematic diagram illustrating an image acquisition scenario provided in this embodiment;

[0024] Figure 6A A flowchart illustrating a method for determining whether an image acquisition process involves injection attacks, provided as an example.

[0025] Figure 6BThis is another flowchart illustrating how to determine whether an image acquisition process involves injection attacks, provided for an embodiment.

[0026] Figure 7 This is a schematic diagram of the identification process for injection attack behavior provided in an embodiment;

[0027] Figure 8 This is a schematic block diagram of an injection attack identification device provided in an embodiment. Detailed Implementation

[0028] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0029] This specification discloses a method and apparatus for identifying injection attack behavior. The application scenarios and technical concepts of the method are first introduced below:

[0030] As mentioned earlier, current defenses against injection attacks in authentication systems such as eKYC rely solely on the image (or video) itself. However, due to the continuous development of AIGC technology, the forged images (or videos) are becoming increasingly realistic, making defense based solely on the image (or video) itself increasingly difficult. To ensure the security of customer information and / or assets for various organizations, it is crucial to provide a new method for identifying and defending against injection attacks.

[0031] In view of this, the inventors propose a method for identifying injection attack behavior. Figure 1 This diagram illustrates an implementation scenario according to an embodiment disclosed in this specification. In this implementation scenario, the process for identifying injection attack behavior is executed on a mobile terminal performing image acquisition. Wherein, as... Figure 1 As shown, the mobile terminal may be equipped with an image acquisition device and a sensor. The sensor may include a first sensor for sensing the motion of the mobile terminal, and a second sensor for sensing the ambient light conditions of the environment in which the mobile terminal is located.

[0032] For example, the first sensor may include, but is not limited to, an accelerometer, a gyroscope, a magnetometer, etc., and the accelerometer may include a linear accelerometer and / or an angular accelerometer; the second sensor may include, but is not limited to, a photodiode, a photoresistor diode, or other sensor capable of sensing ambient light.

[0033] The mobile terminal may have a target program installed, which runs on the terminal's processor. This program is used to acquire data corresponding to the image acquisition process. This data may include sensor data corresponding to sensors in the mobile terminal, and image data identified from images acquired during the image acquisition process. For example, the target program can also be used to identify whether the image acquisition process involves injection attacks based on the acquired data. For example, the target program may exist in the form of an SDK (Software Development Kit). The target program may also exist as a functional plugin for an authentication system.

[0034] For example, when a user needs to authenticate their identity, a mobile terminal can be used to capture images for authentication. In the embodiments of this specification, in order to ensure the security of user privacy information and / or property, the authenticity of the image (sequence) needs to be identified before using the image (sequence) for authentication. That is, it is necessary to identify whether the image capture process involves injection attacks, so as to prevent black market operators from using forged images (sequences) to pass authentication and cause the corresponding user (the user corresponding to the forged image) to suffer loss of privacy information and / or property.

[0035] Specifically, when a user triggers an authentication function (e.g., triggering an image capture button), the target program on the mobile terminal is activated. In response to the user's trigger, the target program on the mobile terminal starts the image capture device to capture an image of a first object. Subsequently, the target program acquires first data corresponding to the image capture process for the first object. This first data includes sensor data, which corresponds to sensors in the mobile terminal, including a first sensor for sensing the movement of the mobile terminal. The program also acquires image data corresponding to the aforementioned image capture process, which is obtained from a first image recognition process and is used to indicate the movement of the mobile terminal. Afterward, the target program uses the first data and the image data to determine whether the image capture process involves an injection attack.

[0036] The above process provides a novel method for identifying injection attacks. Considering that in a real image acquisition process (i.e., an image acquisition process without injection attacks), the movement of the mobile terminal during image acquisition is correlated with the movement of the first object in the acquired image sequence. For example, if the mobile terminal moves to the right relative to the first object, the corresponding image sequence will show that the first object (and / or the background in the image) moves to the left relative to the mobile terminal. Similarly, if the mobile terminal moves closer to the first object, the corresponding image sequence will show that the first object (and / or the background in the image) becomes larger. In other words, the image sequence can also indicate the movement of the mobile terminal. Therefore, in the embodiments provided in this specification, by identifying and determining the correlation between the first data corresponding to the image acquisition process (i.e., sensor data, which can characterize the movement of the mobile terminal during image acquisition) and the image data corresponding to the image acquisition process (which can also indicate the movement of the mobile terminal), for example, by determining whether the movement of the mobile terminal indicated by the two is consistent, it is possible to determine whether the image acquisition process involves injection attacks, thereby achieving the identification of injection attacks.

[0037] Furthermore, in real-world scenarios, sensor data during image acquisition is often difficult to forge. Moreover, forging identical sensor and image data with a consistent relationship within the same image acquisition process is even more challenging. Therefore, the aforementioned process can better detect whether the image acquisition process involves injection attacks, thus improving the security of the identity verification system and ultimately enhancing the privacy and / or property security of users.

[0038] The method for identifying injection attack behaviors provided in this specification will be described in detail below with reference to specific embodiments.

[0039] Figure 2 A flowchart illustrating a method for identifying injection attack behavior according to one embodiment of this specification is shown. This method is executed by an electronic device, which can be any device, equipment, platform, device cluster, etc., with computing and processing capabilities. In some examples, the electronic device can be a mobile terminal that performs image acquisition on a first object, or a server that can communicate with the mobile terminal that performs image acquisition on the first object. Exemplarily, the mobile terminal can be, but is not limited to, terminal devices such as mobile phones and tablet computers.

[0040] In the process of identifying injection attack behaviors, such as Figure 2 As shown, the method includes the following steps S210-S230:

[0041] In step S210, first data corresponding to the image acquisition process of the mobile terminal for the first object is acquired, wherein the first data includes sensor data, which corresponds to the sensor in the mobile terminal, and the sensor is used to sense the motion of the mobile terminal.

[0042] Understandably, given that current identity verification systems (such as eKYC systems) typically verify user identity through card and / or facial recognition, in some examples of this specification, the first object can be, but is not limited to, a card and / or a face. The card can include, but is not limited to, bank cards, various types of identification documents, such as ID cards, driver's licenses, and passports.

[0043] The following describes the injection attack identification method provided in this embodiment, taking an electronic device as a mobile terminal that performs image acquisition targeting the first object as an example. When the electronic device is a server capable of communicating with the mobile terminal that performs image acquisition targeting the first object, the implementation process is similar to that when the electronic device is a mobile terminal; the implementation process is the same as when the electronic device is a mobile terminal and will not be repeated here. Specifically, when the electronic device is a server, the mobile terminal needs to send the acquired first data and image data to the server via the network so that the server can use the first data and image data to determine whether the image acquisition device is involved in an injection attack. Alternatively, the mobile terminal can send the acquired first data and the image sequence corresponding to the image acquisition process to the server so that the server can obtain image data from the image sequence and then use the first data and image data to determine whether the image acquisition device is involved in an injection attack.

[0044] In the case where the electronic device is a mobile terminal that performs image acquisition on a first object, the mobile terminal has a target program installed, such as... Figure 1 As shown, the target program runs on the mobile terminal's processor to perform the identification process for injection attack behaviors. Additionally, as... Figure 1 As shown, the mobile terminal is also equipped with an image acquisition device (such as a camera) for image acquisition when invoked by the target program. The mobile terminal also has multiple types of sensors. These multiple types of sensors include, for example, multiple types of first sensors for sensing the motion of the mobile terminal. Exemplarily, the first sensors may include, but are not limited to, accelerometers, gyroscopes, and magnetometers, and the accelerometer may include linear accelerometers and / or angular accelerometers.

[0045] In some further examples, these multiple types of sensors may also include a second sensor for sensing the ambient light of the environment in which the mobile terminal is located. Subsequently, the relationship between the ambient light conditions corresponding to the second sensor during the image acquisition process and the ambient light conditions indicated in the image data during the image acquisition process can be used to help determine whether the image acquisition process involves injection attacks. This allows for multi-faceted identification of whether the image acquisition process involves injection attacks, thereby improving the accuracy of the identification. The second sensor may include, but is not limited to, sensors capable of sensing ambient light conditions (e.g., intensity) such as photodiodes and photoresistors.

[0046] The following describes the specific identification process for injection attacks. This process can be divided into two stages: the data acquisition stage corresponding to the image acquisition process and the actual identification stage. The data acquisition stage corresponding to the image acquisition process will be described below.

[0047] In some exemplary scenarios, when users need to conduct certain specific transactions online, to ensure the privacy and asset security of users related to those transactions, it is necessary to first authenticate the user. Only after successful authentication can the user proceed with the transaction. To further protect user privacy and asset security, the authenticity of the images (sequences) used for authentication needs to be verified. Specifically, this involves verifying whether the image acquisition process corresponding to authentication involves injection attacks. Accordingly, when a user triggers the authentication function (e.g., by pressing the image acquisition button), the target program on the mobile terminal is triggered to obtain data corresponding to the image acquisition process of the first target.

[0048] Specifically, after the target program is triggered, it can activate the mobile terminal's image acquisition device to start the device and then acquire images. During image acquisition, the target program can provide prompts to the user, instructing them to use a specific acquisition method to acquire images of the first object. Correspondingly, in the actual image acquisition process (i.e., the process without injection attacks), the user controls the mobile terminal to move based on the target program's prompts, allowing the mobile terminal's image acquisition device to acquire images of the first object that meet the specific acquisition method. In the actual image acquisition process, the movement of the mobile terminal is controlled by the user based on the target program's prompts (i.e., the indicated specific acquisition method). In other words, in the actual image acquisition process, the user controls the movement of the mobile terminal based on the target program's prompts (this user action is the user's acquisition action) to use the mobile terminal to acquire images of the first object.

[0049] In the actual image acquisition process, the target program can obtain the image sequence of the first object that is actually acquired by the image acquisition device.

[0050] In image acquisition processes involving injection attacks, mobile terminals are often kept stationary, meaning they are not moved by the user based on prompts from the target program (i.e., instructions on specific acquisition methods) to capture images. At this point, malicious actors (e.g., a program installed on the mobile terminal) replace the actual image sequence captured by the image acquisition device with a forged image sequence, transmitting the forged sequence to the target program to achieve the injection attack. The target program then obtains the forged image sequence.

[0051] During the image acquisition process described above, sensors in the mobile terminal sense the movement of the mobile terminal and the ambient light of its surroundings, and measure corresponding data. Specifically, during the image acquisition process, the first sensor in the mobile terminal senses the movement of the mobile terminal during this process and measures corresponding data (hereinafter referred to as first measurement data), which characterizes the movement of the mobile terminal during this process. Furthermore, the second sensor in the mobile terminal senses the ambient light conditions of the environment in which the mobile terminal is located during this process and measures corresponding data (hereinafter referred to as second measurement data), which characterizes the ambient light conditions of the environment in which the mobile terminal is located during this process. Afterwards, the first sensor (and the second sensor) can send the measured data to the target program.

[0052] Subsequently, the target program can acquire data corresponding to the sensors (including the first sensor and the second sensor) in the mobile terminal.

[0053] In some possible examples, the sensor data in the first data may include: data corresponding to each first sensor and / or pose data of the mobile terminal determined based on the data corresponding to the first sensors, wherein the pose data can indicate the position and attitude of the mobile terminal. For example, the attitude of the mobile terminal can be represented by Euler angles, which include: the rotation angle along the x-axis in three-dimensional space, i.e., pitch; the rotation angle along the z-axis in three-dimensional space, i.e., roll; and the rotation angle along the y-axis in three-dimensional space, i.e., yaw. The first data may also include data corresponding to the second sensor, such as ambient light data. Specifically, the first data may include sensor data corresponding to multiple acquisition timestamps. These multiple acquisition timestamps all fall within the time range corresponding to the image acquisition process.

[0054] In some examples, the pose data may be determined by the target program based on the data corresponding to the first sensor; in other examples, the pose data may also be obtained by the target program from the first program, wherein the first program may determine the corresponding pose data based on the data corresponding to the first sensor in the image acquisition process, and then provide it to the target program.

[0055] The following section uses an example of a linear accelerometer and a gyroscope as the first sensors to illustrate the determination of pose data. The linear accelerometer measures the linear acceleration of the mobile terminal along the x, y, and z axes in three-dimensional space, while the gyroscope measures the rotational speed of the mobile terminal along the x, y, and z axes. By integrating the data from the linear accelerometer and gyroscope, the pose changes of the mobile terminal during image acquisition relative to its initial state (the position and orientation of the mobile terminal when the image acquisition device begins acquiring images) can be estimated. Therefore, any correlation technique can be used to determine a series of pose data of the mobile terminal during image acquisition, i.e., the pose data corresponding to each acquisition timestamp, based on the estimated pose changes.

[0056] Specifically, in some possible examples, where the first data is real data corresponding to the actual image acquisition process (i.e., the image acquisition process does not involve injection attacks), the acquired sensor data may include data directly measured by multiple types of sensors in the mobile terminal (e.g., including the first and second sensors) (i.e., the aforementioned first measurement data) and / or the pose data of the mobile terminal determined based on the data measured by the first sensor; it may also include data directly measured by the second sensor (i.e., the aforementioned second measurement data). In this case, this type of sensor data conforms to the user's behavior of controlling the movement of the mobile terminal based on the prompts of the target program (also known as the user's acquisition behavior), that is, it conforms to the user's acquisition behavior corresponding to the specific acquisition method adopted by the user during the image acquisition process.

[0057] In some other possible examples, where the first data is injection data corresponding to the image acquisition process, i.e., the image acquisition process involves injection attack behavior, the acquired sensor data may include: forged data corresponding to multiple types of sensors (e.g., including the first sensor and the second sensor) (i.e., forged data measured by the first sensor), and / or forged pose data of the mobile terminal determined based on the forged data corresponding to the first sensor; it may also include forged data corresponding to the second sensor (i.e., forged data measured by the second sensor).

[0058] In some specific examples, where the image acquisition process involves injection attacks, after the sensors in the mobile terminal obtain measurement data, during the transmission of this data to the target program, the malicious actors intercept the sensor data and replace it with forged data corresponding to multiple sensor types. In another implementation, the target program uses this forged data to determine the mobile terminal's pose data, thus obtaining forged sensor data. Alternatively, the forged data includes both forged measurement data for the mobile terminal's sensors and the corresponding pose data, allowing the target program to directly obtain the forged sensor data.

[0059] In some other examples, where the image acquisition process involves injection attacks, the sensor data acquired by the target program may be data actually measured by the mobile terminal's sensors and / or pose data obtained based on that data. However, this type of sensor data may not conform to the user's behavior of controlling the mobile terminal based on the target program's prompts, i.e., it may not conform to the user's acquisition behavior corresponding to the specific acquisition method used during the image acquisition process. For example, during image acquisition involving injection attacks, the mobile terminal may be stationary (i.e., the user does not control the mobile terminal to move according to the target program's prompts, i.e., the user's acquisition behavior). In this case, the data measured by the sensors in the mobile terminal may all be 0.

[0060] In some other examples, where the image acquisition process involves injection attacks, the target program may be unable to obtain the sensor's measurement data. In such cases, the target program can directly determine that the image acquisition process involves injection attacks and will not continue executing subsequent processes.

[0061] Each time the target program acquires data corresponding to a sensor in the mobile terminal, it can simultaneously record the timestamp corresponding to each acquired data. This timestamp can be the acquisition timestamp mentioned later. Accordingly, the first data can include sensor data corresponding to multiple acquisition timestamps during the image acquisition process. In some examples, the sensor data may be data corresponding to the time period from the start to the end of the image acquisition process, or it may be data corresponding to timestamps at preset intervals during the time period from the start to the end of the image acquisition process.

[0062] Understandably, when a mobile terminal has multiple sensors, the data measurement frequencies of different sensors may be different. After the target program obtains the data corresponding to multiple sensors, it can use any data alignment method in related technologies to align the data corresponding to multiple sensors.

[0063] In some implementations, the aforementioned first data may further include acquisition method data, which is used to instruct the user to use a specific acquisition method during image acquisition. It is understood that this acquisition method data, used to instruct the user to use a specific acquisition method during image acquisition, is generated by the target program and cannot be forged by malicious actors. Accordingly, to better improve the accuracy of identifying whether the image acquisition process involves injection attacks, the first data may also include acquisition method data. Specifically, the first data may include acquisition method data corresponding to multiple acquisition timestamps during the image acquisition process.

[0064] The following describes the process of acquiring data during image acquisition:

[0065] In some examples, after the target program detects that the image acquisition device has been activated, it begins the image acquisition process for the first object. During this image acquisition process, the target program may first output a prompt message to remind the user to perform a frontal image acquisition of the first object (e.g., ...). Figure 3 As shown in the diagram, correspondingly, during the actual image acquisition process, the user controls the movement of the mobile terminal based on the first prompt information, thereby enabling the mobile terminal's image acquisition device to acquire a frontal image of the first object. After acquiring the frontal image of the first object, the target program records the end time of the frontal acquisition sub-process of the first object (hereinafter referred to as the first end time). For example, the first end time can be the time when the target program detects that a frontal image of the first object that meets the preset conditions has been acquired, or it can be the time when it detects that the user triggers the frontal acquisition end button.

[0066] Subsequently, the target program can generate acquisition mode data based on the startup time and the first end time of the image acquisition device, and include it in the first data corresponding to the image acquisition process. For example, between the startup time and the first end time of the image acquisition device (i.e., the frontal acquisition sub-process of the first object), the acquisition mode data includes data indicating the frontal acquisition mode. That is, between the startup time and the first end time of the image acquisition device, there are corresponding timestamps for sensor data acquisition, and the corresponding acquisition mode data includes data indicating the frontal acquisition mode.

[0067] For example, the aforementioned frontal acquisition can refer to an acquisition method in which the first object is not tilted. For instance, it can mean that the plane where the first object is located is parallel to the plane where the display screen of the mobile terminal is located, or it can be considered that the plane where the first object is located is parallel to the plane where the image acquisition device of the mobile terminal is located. This parallelism can mean that the angle between the plane where the first object is located and the plane where the display screen of the mobile terminal is located is less than a preset angle threshold.

[0068] Considering that during an injection attack (i.e., the image acquisition process involving an injection attack), malicious actors may forge sensor data and inject this forged data into the target program to deceive it into believing that the forged sensor data is the measurement data (and / or the determined pose data of the mobile terminal) measured by the mobile terminal's sensors during the image acquisition process; and considering that the acquisition method data is the data that instructs the user to use a specific acquisition method during the image acquisition process (i.e., the aforementioned prompt information), it is generated by the target program and cannot be forged by malicious actors. Therefore, to increase the complexity of the sensor data forged by malicious actors during the image acquisition process, making it impossible for them to forge sensor data that satisfies the user's specific acquisition method, thereby improving the detection rate and accuracy of injection attacks and enhancing defense against injection attacks, this specification's embodiments design a more random and complex acquisition method, and / or a random vibration acquisition method.

[0069] In some possible examples, exemplaryly, the acquisition method data may include: data indicating a frontal acquisition method and / or data indicating a non-frontal acquisition method, the non-frontal acquisition method including: a random multi-tilt angle acquisition method. Specifically, the random multi-tilt angle acquisition method may include instructing the user to acquire an image of a first object with specified tilt information (e.g., ...). Figure 3 (As shown in "Acquisition of specified tilt information"). This specified tilt information is randomly generated.

[0070] For example, in the case where the first object is a face, the specified tilt information may include information indicating that images are to be captured for the left side of the face and / or for the right side of the face.

[0071] For example, when the first object is a card, the specified tilt information may include an indication of the card's tilt direction and / or tilt angle. This tilt direction and / or tilt angle are randomly generated to ensure randomness, thereby increasing the difficulty and complexity for black market operators to forge sensor data that meets the user's specific data collection needs. For example, the tilt direction may include, but is not limited to, at least one of the following: tilting upwards, tilting downwards, tilting to the left, and tilting to the right. For example, the tilt angle may include, but is not limited to, at least one of the following: 15 degrees, 30 degrees, 40 degrees, etc.

[0072] Accordingly, in some possible examples, during the image acquisition process of the first object, after the target program detects the end of the frontal acquisition sub-process of the first object, it can continue to output a second prompt message to prompt the user to perform non-frontal acquisition of the first object, i.e., to use a non-frontal acquisition method for image acquisition. The second prompt message may include randomly generated information instructing the user to perform non-frontal acquisition of the first object with a certain tilt setting.

[0073] Correspondingly, during the actual image acquisition process, the user can control the movement of the mobile terminal based on the second prompt information, so that the image acquisition device of the mobile terminal can acquire a non-frontal image of the first object that satisfies the second prompt information.

[0074] Subsequently, a non-frontal image of the first object satisfying the second prompt information is acquired, and the target program records the end time of the non-frontal acquisition sub-process of the first object (hereinafter referred to as the second end time). The second end time can be the time when the target program detects that a non-frontal image of the first object satisfying the preset conditions has been acquired, or the time when it detects that the user has triggered the non-frontal acquisition end button. Subsequently, the target program can generate acquisition method data based on the aforementioned image acquisition device startup time, first end time, second end time, and tilt information included in the second prompt information, and incorporate it into the first data corresponding to the image acquisition process.

[0075] For example, between the first end time and the second end time of the image acquisition device (i.e., the non-frontal acquisition sub-process of the first object), the acquisition mode data may include data indicating the non-frontal acquisition mode, and may also include the aforementioned specified tilt information. That is, each of the time stamps corresponding to the sensor data acquisition between the first end time and the second end time of the image acquisition device includes data indicating the non-frontal acquisition mode and / or specified tilt information.

[0076] Understandably, the non-frontal acquisition method is optional, meaning it can occur randomly. In other words, during the image acquisition process for different first objects, the target program can randomly instruct the user whether to use the non-frontal acquisition method, so that the image acquisition device can acquire the corresponding non-frontal image of the first object.

[0077] In some other possible examples, considering that the mobile terminal has a preset vibrator, the vibration of the preset vibrator will have a certain impact on the measurement data obtained by the mobile terminal's sensors. To further increase the difficulty of obtaining sensor data corresponding to the image acquisition process forged by black market operators and improve the detection rate and accuracy of injection attacks, the target program can also be set as follows: during image acquisition, the mobile terminal (the target program in the program) randomly generates a first instruction, which instructs the preset vibrator in the mobile terminal to vibrate; correspondingly, the aforementioned first data can also include data indicating whether the mobile terminal vibrates. It is understood that during image acquisition, the target program of the mobile terminal can randomly generate a first instruction to randomly instruct the preset vibrator in the mobile terminal to vibrate, so that the mobile terminal vibrates randomly (e.g., ...). Figure 3 As shown), the first instruction can carry information indicating the vibration duration, wherein the vibration duration can be randomly generated to instruct the preset vibrator in the mobile terminal to vibrate for an irregular duration, so as to better improve randomness and increase the complexity and difficulty of the sensor data corresponding to the black market forged image acquisition process.

[0078] Understandably, during the aforementioned image acquisition process, the mobile terminal's sensors are constantly sensing the mobile terminal's motion (and the ambient light conditions of the environment in which the mobile terminal is located), and measuring its data (such as...). Figure 3 The "sensor measurement data" shown is used throughout the entire image acquisition process.

[0079] Accordingly, in the above example, the first data includes sensor data corresponding to multiple acquisition timestamps during the image acquisition process, acquisition method data, and data indicating whether the mobile terminal vibrates.

[0080] After obtaining the first data corresponding to the image acquisition process, in step S220, the image data corresponding to the image acquisition process is obtained, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal.

[0081] In some possible examples, the image data includes image data identified from the first images corresponding to the aforementioned multiple acquisition timestamps.

[0082] In some examples, changes in the pose and / or other attributes of the first object across a series of images (i.e., first images corresponding to multiple acquisition timestamps) can indicate relative changes between the first object and the mobile terminal, and consequently, indicate the movement of the mobile terminal. For instance, if an image indicates that the first object is moving to the left relative to the mobile terminal, it can correspondingly indicate that the mobile terminal is moving to the right; if an image indicates that the first object is larger (the area of ​​the first object in the image occupies a larger proportion of the image), it can indicate that the mobile terminal is moving closer to the first object.

[0083] In some other examples, the motion trajectory of the background (including objects other than the first object) in a series of images (i.e., the first image corresponding to multiple acquisition timestamps) can also indicate the motion of the mobile terminal. Accordingly, the image data may include the relevant feature data of the first object identified from the first image (i.e., subsequent object feature data) and / or the relevant feature data of the background identified from the first image (i.e., subsequent background feature data).

[0084] Understandably, the process of obtaining image data from any first image is similar for the target program. The following explanation uses any first image as an example to illustrate the process of obtaining its image data.

[0085] In some possible examples, such as Figure 4 As shown, step S220 may include step 11:

[0086] In step 11, if the first image is found to contain the first object, the object feature data corresponding to the first object is detected from the first image and included in the image data corresponding to the first image.

[0087] In some implementations, the first image corresponding to the image acquisition process can be an image actually acquired by the image acquisition device of the mobile terminal; or the first image corresponding to the image acquisition process can be a forged image injected by black market actors.

[0088] In some exemplary scenarios, after the image acquisition device is triggered and started by the target program, the mobile terminal's display screen can show an image preview page, which can display the image acquisition frame, such as... Figure 5 As shown, this instructs the user to position the first object appropriately so that it can be fully displayed within the image acquisition frame. When acquiring an image, the image acquisition device can capture an area larger than the image acquisition frame to facilitate image data acquisition. For example... Figure 5 As shown, the image acquisition device can acquire the image corresponding to the first frame. For example, the image preview page can also display the aforementioned prompt information to guide the user to acquire the image using a specific acquisition method.

[0089] Specifically, in a genuine image acquisition process (i.e., an image acquisition process without injection attacks), after the image acquisition device is started, the target program can obtain images from the image acquisition device in real time and use them as the first image. In an image acquisition process involving injection attacks, the cybercriminal will intercept the images (sequences) transmitted by the image acquisition device to the target program, replace the intercepted images transmitted by the image acquisition device to the target program with forged images (sequences), and transmit them to the target program. The target program then obtains the images (sequences) and uses them as the first image.

[0090] Subsequently, the target program can detect whether the first image it has acquired contains a first object. If the first image contains a first object, the program can detect the object feature data corresponding to the first object from the first image and classify it into the image data corresponding to the first image.

[0091] For example, a pre-trained first object recognition network or a traditional image recognition algorithm can be used to detect whether the first image contains a first object. In some examples, the first object recognition network can be a neural network pre-trained based on sample images containing the first object and their corresponding labels, and this neural network can be a CNN (Convolutional Neural Network).

[0092] In some possible implementations, the aforementioned process of detecting object feature data corresponding to the first object from the first image may include: after detecting the first image using a pre-trained first target recognition network, the first target recognition network, upon determining that the first image contains the first object, may output the first position information of the first object in the first image, which is the region position of the first object identified by the first target recognition network in the first image; subsequently, the region position information of the image region where the first object is located in the first image may be determined based on the first position information.

[0093] In one implementation, the first location information is used as the region location information of the image region where the first object is located in the first image. In another implementation, since the first location information has relatively low precision, in order to obtain more accurate and precise region location information of the first object, after determining the first location information, the target program extracts the region image corresponding to the first location information from the first image; then, it continues to detect the region image to detect the region location information of the image region where the first object is located, and classifies the region location information into the object feature data corresponding to the first object.

[0094] Specifically, a pre-trained second target recognition network or a traditional image recognition algorithm can be used to detect the regional location information of the image region where the first object is located from the regional image. The second target recognition network can be a neural network pre-trained based on a sample image containing the first object and its corresponding label indicating the location region of the first object in the sample image. This neural network can be a CNN (Convolutional Neural Network).

[0095] Next, the target program on the mobile terminal inputs the first image, or the aforementioned region image cropped from the first image, into a pre-trained attribute recognition network to obtain attribute data corresponding to the first object. This attribute data is then incorporated into the object feature data corresponding to the first object. The attribute data may include, but is not limited to, the Euler angles, sharpness value, and brightness value corresponding to the first object. The attribute recognition network can be a neural network pre-trained based on labels containing sample images of the first object and their corresponding indicator sample images showing the attribute data of the first object. It can be a neural network of any structure, such as a convolutional neural network or a transformer-based network.

[0096] After obtaining the region location information of the image area where the first object is located in the first image, the target program can calculate the proportion of the image area where the first object is located in the first image (referred to as the first proportion) based on this region location information. Then, based on the first proportion, it determines the first estimated distance between the first object and the mobile terminal, and incorporates the first estimated distance into the object feature data corresponding to the first object. It is understood that the larger the proportion of the image area where the first object is located in the first image, the smaller the first estimated distance between the first object and the mobile terminal. For example, a pre-defined correspondence between the proportions of the image area where the first object is located in its respective image and the estimated distances can be established, so that this correspondence can be used to determine the first estimated distance between the first object and the mobile terminal based on the first proportion.

[0097] Accordingly, in some possible examples, the object feature data includes at least one of the following types of data: regional location information of the image region where the first object is located in the first image, the Euler angles, sharpness value, and brightness value corresponding to the first object, and a first estimated distance between the first object and the mobile terminal. The first estimated distance is determined based on the proportion of the image region where the first object is located in the first image. The Euler angles, sharpness value, and brightness value corresponding to the first object are determined based on the first image and a pre-trained attribute recognition network. A higher brightness value corresponding to the first object can also characterize a stronger ambient light in the environment where the mobile terminal is located.

[0098] It is understood that the above are merely exemplary examples of object feature data and do not constitute a limitation on object feature data. Any feature data that can indicate the relative relationship between the first object and the mobile terminal can be used to identify the motion of the mobile terminal from the image and included in the object feature data. For example, object feature data may also include data indicating whether the first object contained in the first image is complete, etc.

[0099] In some possible examples, when the first object is a card, the first object can be held by the user for image acquisition, or placed on a flat surface (such as a desktop) (i.e., not held by the user) for image acquisition. When the first object is held by the user, i.e., in a handheld state, it is possible that during image acquisition, the user controls the movement of the first object and simultaneously controls the movement of the mobile terminal so that the image acquisition device can acquire an image that satisfies the prompt information output by the target program (i.e., satisfies the specific acquisition method adopted by the user).

[0100] In some other possible examples, where the first object is a face, it is also possible that during the image acquisition process, the face is moved by the user based on prompts from the target program, while the user controls the movement of the mobile terminal so that the image acquisition device can acquire an image that satisfies the prompts output by the target program (i.e., satisfies the specific acquisition method adopted by the user).

[0101] In the above situation, if the movement of the mobile terminal is indicated by changes in the image of a face or a card in a handheld state (such as changes in pose data, i.e., changes in Euler angles and region location information in object feature data), the movement of the mobile terminal indicated by the image is not accurate enough. This may lead to an error in matching the movement of the mobile terminal indicated by the image data with the movement of the mobile terminal indicated by the sensor data actually measured by the mobile terminal's sensor during the corresponding process.

[0102] Given the above, in order to ensure the accuracy of identifying whether the image acquisition process involves injection attacks, in some possible examples, such as... Figure 4 As shown, step S220 may further include steps 12-13:

[0103] In step 12, if the first image is detected to contain the first object, the image region containing the first object is removed from the first image to obtain the background region image corresponding to the first image.

[0104] In step 13, background feature data is extracted from the background region image and incorporated into the image data corresponding to the first image. The background feature data is used to calculate the motion of the first image relative to its previous frame image.

[0105] In some possible examples, when the target program detects that the first image contains the first object, it can extract the image region containing the first object from the first image based on the previously identified region location information of the image region where the first object is located, thus obtaining the background region image corresponding to the first image. Then, background feature data is extracted from the background region image and used as the image data corresponding to the first image. This background feature data is used to calculate the motion of the first image relative to its previous frame.

[0106] In some other possible examples, if the target program detects that the first object contained in the first image is a card, it can continue to identify whether the first object is held in a hand or placed on a table. Then, if the first object (e.g., a card) is detected to be held in a hand, or if the first object contained in the first image is detected to be a face, steps 12-13 can be executed to reduce the computational consumption of the mobile terminal to some extent.

[0107] In some implementations, the target program can use a pre-trained state recognition network to detect and identify whether the first object is in a handheld state or in a state placed on a plane (or a non-handheld state) using a first image or a region image cropped from the first image. This state recognition network can be a network pre-trained based on specified sample images and their corresponding labels; such a network could be a convolutional neural network, a transformer-based network, etc.

[0108] The specified sample images include images containing a first object in a handheld state and images containing a first object in a non-handheld state; for images containing a first object in a handheld state, the corresponding label is a label indicating that the first object is in a handheld state; for images containing a first object in a non-handheld state, the corresponding label is a label indicating that the first object is in a non-handheld state.

[0109] The aforementioned process of extracting background feature data from a background region image may include: extracting background feature data from the background region image using a specified feature extraction algorithm. For example, the specified feature extraction algorithm may be, but is not limited to, SIFT (Scale-Invariant Feature Transform), optical flow tracing, ORB (Oriented Fast and Rotated BRIEF), or Speeded Uprobust Features (SURF), etc. For example, the background feature data may include feature points extracted based on the specified feature extraction algorithm. Furthermore, the background feature data may also include background brightness values, which is also possible. These background brightness values ​​can be determined using any image brightness determination algorithm in related technologies. These background brightness values ​​can, to some extent, indicate the ambient light conditions of the environment in which the mobile terminal is located; a higher background brightness value indicates stronger ambient light in the environment in which the mobile terminal is located.

[0110] In some possible examples, if the first image does not contain the first object, the value of its corresponding object feature data can be set to a specified numerical value (e.g., 0). In some possible implementations, if the first image does not contain the first object, the brightness value in the object feature data can also be set based on the brightness value of the entire first image, and the values ​​of other data in the object feature data besides the brightness value can be set to specified numerical values.

[0111] In some other possible examples, such as Figure 4 As shown, step S220 may further include step 14:

[0112] In step 14, if the first image does not contain the first object, background feature data is extracted from the first image and incorporated into the image data corresponding to the first image. The background feature data is used to calculate the motion of the first image relative to its previous frame image.

[0113] In the above example, if the target program detects that the first image does not contain the first object, it can continue to extract background feature data from the first image and incorporate it into the image data corresponding to the first image. This allows the program to calculate the motion of the first image relative to its previous frame based on the specified feature data, thereby indicating the motion of the mobile terminal. The implementation principle of extracting background feature data from the first image is similar to that of step 13 described above, and its implementation method can be found in the implementation method of step 13 described above, which will not be repeated here.

[0114] After obtaining the first data and image data, in step S230, the first data and image data are used to determine whether the image acquisition process involves injection attack behavior.

[0115] Considering that in the actual image acquisition process, the target program obtains a sequence of images (first images corresponding to multiple acquisition timestamps) from the image acquisition device, and the poses of the background and / or the first object in each image can be characterized by changes, which can indicate the movement of the mobile terminal during the image acquisition process; and the pose of the mobile terminal generally changes during the image acquisition process, and correspondingly, the data actually acquired by the mobile terminal's sensor (e.g., the first sensor) also changes, and the changes in the poses of the background and / or the first object represented by the image match the changes in the actual sensor data of the mobile terminal's sensor during the image acquisition process.

[0116] In addition, during the actual image acquisition process, the images can also represent the changes in the environment of the mobile terminal (e.g., ambient light conditions), and the data actually collected by the mobile terminal's sensors (e.g., the second sensor) can also represent the changes in the environment of the mobile terminal (e.g., ambient light conditions), and the changes represented by the two are matched.

[0117] In the image acquisition process involving injection attacks, one implementation involves the image sequence corresponding to the image acquisition process being forged by black market actors. The sensor data in the first data corresponding to the image acquisition process is obtained based on the actual data collected by the mobile terminal's sensors, which can indicate the actual movement of the mobile terminal (and the actual ambient light conditions of the mobile terminal's environment). In this case, the probability of the movement of the mobile terminal indicated by the image data corresponding to each image in the image sequence matching the movement of the mobile terminal indicated by the sensor data in the first data is low (and the probability of the ambient light conditions of the mobile terminal's environment indicated by the image data corresponding to each image in the image sequence matching the actual ambient light conditions of the mobile terminal's environment indicated by the sensor data in the first data is also low).

[0118] In one implementation of image acquisition involving injection attacks, the image sequence corresponding to the image acquisition process and the sensor data in the first data are both forged by black market actors. The probability of the motion of the mobile terminal indicated by the two is low (and the probability of the ambient light conditions of the environment where the mobile terminal is located, indicated by the image data corresponding to each image in the image sequence, matching the actual ambient light conditions of the environment where the mobile terminal is located, indicated by the sensor data in the first data is also low). Furthermore, the acquisition method data in the first data is generated by the target program itself and cannot be forged. Accordingly, the probability of the motion of the mobile terminal indicated by the image data corresponding to each image in the image sequence and the motion of the mobile terminal indicated by the sensor data in the first data matching the acquisition method data in the first data is low.

[0119] Given the above, the target program can use the first data and image data to determine whether the image acquisition process involves injection attacks.

[0120] Specifically, after acquiring the first data and image data, the target program can determine, at least, the motion of the mobile terminal indicated by each of the first data and image data. In some possible examples, the image data may include object feature data and / or background feature data identified from each of the first images corresponding to multiple acquisition timestamps; the motion of the mobile terminal can be determined based on the object feature data and / or background feature data in the image data corresponding to each first image.

[0121] Taking object feature data as an example, based on the region location information and Euler angles of the first object in the object feature data corresponding to the i-th first image, and the region location information and Euler angles of the first object in the object feature data corresponding to the (i-1)-th first image, the pose data corresponding to the i-th first image of the mobile terminal (hereinafter referred to as image pose data) can be determined, that is, the image pose data corresponding to the acquisition timestamp t_i of the i-th first image. The image pose data in the aforementioned example may include rotation angles and displacement vectors; the rotation angle indicates the rotation angle (including Euler angles) of the mobile terminal relative to the mobile terminal at time t_i at time t_(i-1) in the x, y, and z axes of three-dimensional space; the displacement vector indicates the distance of displacement of the mobile terminal relative to the mobile terminal at time t_i at time t_(i-1) in the x, y, and z axes of three-dimensional space.

[0122] Taking background feature data as an example, the background feature data in the object feature data corresponding to each first image includes feature points extracted based on a specified feature extraction algorithm. Correspondingly, brute-force matching or a fast approximate nearest neighbor search based on FLANN (Fast Library for Approximate Nearest Neighbors) can be used to match multiple feature point pairs between the i-th and (i-1)-th first images. Then, the RANSAC (random sample consensus) algorithm or other possible filtering algorithms can be used to remove potentially abnormal feature point pairs. Next, based on the filtered feature point pairs, the essential matrix corresponding to the i-th and (i-1)-th first images is calculated. By decomposing the essential matrix, the image pose data of the i-th first image relative to the (i-1)-th first image is obtained, i.e., the image pose data corresponding to the i-th first image, specifically the image pose data corresponding to the acquisition timestamp t_i of the i-th first image.

[0123] It should be noted that when i is 1, the values ​​of each data item in the relative pose data of the first image relative to the non-existent zeroth image can be recorded as 0.

[0124] The sensor data corresponding to each of the multiple acquisition timestamps in the first data may include pose data corresponding to each acquisition timestamp; then, it can be determined whether the image acquisition process involves injection attack behavior by determining whether the motion trend of the mobile terminal indicated by the image pose data corresponding to the multiple acquisition timestamps matches the motion trend of the mobile terminal indicated by the pose data in the sensor data corresponding to the multiple acquisition timestamps.

[0125] Specifically, if the motion trend of the mobile terminal indicated by the image pose data corresponding to each acquisition time stamp matches the motion trend of the mobile terminal indicated by the pose data in the sensor data corresponding to multiple acquisition time stamps, then the image acquisition process is determined not to involve injection attack behavior; conversely, if the motion trend of the mobile terminal indicated by the image pose data corresponding to each acquisition time stamp does not match the motion trend of the mobile terminal indicated by the pose data in the sensor data corresponding to multiple acquisition time stamps, then the image acquisition process is determined to involve injection attack behavior.

[0126] Subsequently, in some examples, after determining that the image acquisition process does not involve injection attacks, the first image acquired during the image acquisition process can be used to authenticate the user. In other examples, after determining that the image acquisition process involves injection attacks, the user can be prompted to re-acquire the image; on the other hand, to protect the user's privacy and / or property security, the user's account can be locked, and the user can be notified of the abnormality of their account through other contact methods set by the user, thus helping the user avoid risks.

[0127] In some other examples, the ambient light conditions of the mobile terminal's environment can be determined based on the first data and image data. Specifically, the first brightness change value corresponding to each first image is determined based on the brightness value of the first object in the object feature data of the image data corresponding to each first image and / or the background brightness value in the background feature data. For example, based on the brightness value of the first object in the object feature data corresponding to the i-th first image and the brightness value of the first object in the object feature data corresponding to the (i-1)-th first image, the first brightness change value of the i-th first image relative to the (i-1)-th first image is determined, that is, the first brightness change value corresponding to the acquisition timestamp t_i of the i-th first image.

[0128] Based on the data from the second sensor among multiple acquisition timestamps, the second brightness change value corresponding to each acquisition timestamp is determined. For example, based on the data from the second sensor corresponding to acquisition timestamp t_i and the data from the second sensor corresponding to acquisition timestamp t_i-1, the second brightness change value of acquisition timestamp t_i relative to acquisition timestamp t_i-1 is determined, that is, the second brightness change value corresponding to acquisition timestamp t_i.

[0129] Subsequently, by determining whether the brightness value change indicated by the first brightness change value corresponding to the acquisition timestamp of each first image matches the brightness value change indicated by the second brightness change value corresponding to the acquisition timestamp t_i, and by determining whether the motion trend of the mobile terminal indicated by the image pose data corresponding to multiple acquisition timestamps matches the motion trend of the mobile terminal indicated by the pose data in the sensor data corresponding to multiple acquisition timestamps, it is jointly determined whether the image acquisition process involves injection attack behavior, so as to better improve the identification accuracy of injection attack behavior.

[0130] For example, if the brightness value change indicated by the first brightness change value corresponding to the acquisition timestamp of each first image matches the brightness value change indicated by the second brightness change value corresponding to the acquisition timestamp t_i, and if the motion trend of the mobile terminal indicated by the image pose data corresponding to multiple acquisition timestamps matches the motion trend of the mobile terminal indicated by the pose data in the sensor data corresponding to multiple acquisition timestamps, it can be determined that the image acquisition process involves injection attack behavior; if any one of these aspects does not match, it can be determined that the image acquisition process involves injection attack behavior.

[0131] In some other possible examples, a pre-trained machine learning network can be used in conjunction with the first data and image data to determine whether the image acquisition process involves injection attacks. Specifically, step S230 can include step 21:

[0132] In step 21, using the first data and image data, an injection attack identification network is used to determine whether the image acquisition process involves injection attack behavior. The injection attack identification network is a network pre-trained based on the sample data and corresponding label data of each sample acquisition process. The sample data includes the sample sensor data and sample image data corresponding to the corresponding sample acquisition process, and the label data is used to indicate whether the corresponding sample acquisition process involves injection attack behavior.

[0133] In some possible examples, the first data includes sensor data corresponding to multiple acquisition timestamps, and the image data includes image data identified from each first image corresponding to multiple acquisition timestamps; the aforementioned injection attack identification network includes a matching degree identification subnetwork;

[0134] like Figure 6A As shown, step 21 may include steps 211-214:

[0135] In step 211, based on the sensor data corresponding to each acquisition timestamp and the sensor data corresponding to the previous acquisition timestamp, the first change situation corresponding to each acquisition timestamp is determined. In this step, taking acquisition timestamp t_i as an example, specifically, the difference between the sensor data corresponding to acquisition timestamp t_i and the sensor data corresponding to acquisition timestamp t_i-1 can be calculated as the first difference corresponding to acquisition timestamp t_i. Based on the first difference corresponding to acquisition timestamp t_i, the first change situation corresponding to acquisition timestamp t_i is determined, where the first change situation corresponding to acquisition timestamp t_i includes the first difference corresponding to acquisition timestamp t_i.

[0136] In some other examples, the first change corresponding to each acquisition time stamp can be determined based on the sensor data and acquisition method data corresponding to each acquisition time stamp, and the sensor data and acquisition method data corresponding to the previous acquisition time stamp. The implementation principle of determining the first change corresponding to each acquisition time stamp is similar to that of step 211 above, and its implementation process can be found in the implementation process of step 211 above, which will not be repeated here.

[0137] In step 212, based on the image data corresponding to each acquisition timestamp and the image data corresponding to the previous acquisition timestamp, a second change situation corresponding to each acquisition timestamp is determined. For example, the multi-dimensional feature data included in the image data is quantized data. In this step, taking acquisition timestamp t_i as an example, specifically, the difference between the image data corresponding to acquisition timestamp t_i and the image data corresponding to acquisition timestamp t_i-1 can be calculated as the second difference corresponding to acquisition timestamp t_i. Based on the second difference corresponding to acquisition timestamp t_i, the second change situation corresponding to acquisition timestamp t_i is determined, wherein the second change situation corresponding to acquisition timestamp t_i includes the second difference corresponding to acquisition timestamp t_i.

[0138] In step 213, based on the first and second changes corresponding to each acquisition time stamp, the predicted matching degree value between the sensor data and image data corresponding to each acquisition time stamp is obtained through the matching degree recognition sub-network.

[0139] In some examples, the matching degree recognition subnetwork includes an encoder and a classifier. Accordingly, the first and second changes corresponding to each collection time stamp can be concatenated to obtain the concatenated changes corresponding to each collection time stamp. For example, the first change X1t_i corresponding to collection time stamp t_i is represented as [a, b, c, d], and the second change X2t_i corresponding to collection time stamp t_i is represented as [e, f, g]. The resulting concatenated changes X1t_i+X2t_i corresponding to collection time stamp t_i can be represented as [a, b, c, d, e, f, g].

[0140] Next, the splicing changes corresponding to each acquisition time stamp are input into the encoder of the matching degree recognition subnetwork. The encoder processes the splicing changes corresponding to each acquisition time stamp to obtain the encoded representation (hereinafter referred to as the first encoded representation) corresponding to each acquisition time stamp. Then, the first encoded representation corresponding to each acquisition time stamp is input into the classifier of the matching degree recognition subnetwork. The classifier processes the first encoded representation corresponding to each acquisition time stamp to obtain the predicted matching degree value between the sensor data and image data corresponding to each acquisition time stamp.

[0141] In some examples, the matching degree recognition subnetwork may include two encoders and one classifier. One encoder corresponds to a first change and is called the first encoder; the other encoder corresponds to a second change and is called the second encoder. Accordingly, the first change corresponding to each acquisition timepoint is input into the first encoder to obtain an encoded representation of the first change corresponding to each acquisition timepoint (hereinafter referred to as the second encoded representation); and the second change corresponding to each acquisition timepoint is input into the second encoder to obtain an encoded representation of the second change corresponding to each acquisition timepoint (hereinafter referred to as the third encoded representation). Then, the second and third encoded representations corresponding to each acquisition timepoint are concatenated to obtain a concatenated encoded representation for each acquisition timepoint. Next, the concatenated encoded representation for each acquisition timepoint is input into the classifier to obtain the predicted matching degree value between the sensor data and image data corresponding to each acquisition timepoint.

[0142] The implementation principle of splicing the second and third coding representations corresponding to each collection timestamp is the same as the aforementioned implementation principle of splicing the first and second change situations corresponding to each collection timestamp. The implementation process is the same as the aforementioned implementation process of splicing the first and second change situations corresponding to each collection timestamp, and will not be repeated here.

[0143] In some implementations, the matching degree recognition subnetwork can be implemented by any machine learning network, including but not limited to transformer-based networks, recognition networks based on SVM (Support Vector Machine) algorithms, recognition networks based on random forest algorithms, and recognition networks based on boost algorithms.

[0144] In step 214, based on each predicted matching degree value, it is determined whether the image acquisition process involves injection attack behavior. In some examples, if the number of predicted matching degree values ​​below a preset threshold exceeds a first preset number, or if the ratio between the number of predicted matching degree values ​​below the preset threshold and the total number of predicted matching degree values ​​exceeds a preset proportion, then it can be determined that the image acquisition process involves injection attack behavior. Conversely, if the ratio does not exceed the preset threshold, then it is determined that the image acquisition process does not involve injection attack behavior.

[0145] In some further examples, if a second preset number of predicted matching values ​​below a preset threshold appear consecutively in each predicted matching value, it can be determined that the image acquisition process involves injection attacks. If a second preset number of predicted matching values ​​below a preset threshold do not appear consecutively in each predicted matching value, and the number of predicted matching values ​​below the preset threshold in each predicted matching value does not exceed a third preset number, it can be determined that the image acquisition process does not involve injection attacks.

[0146] Considering that the first image corresponding to the image acquisition process may not include the first object, or may include the first object and the first object is a face, or may include the first object and the first object is a handheld card, the first object is prone to being moved. If the first object is moved, and the mobile terminal is also moved, the movement of the mobile terminal determined using the movement of the first object presented in the image may be inaccurate. Therefore, to ensure the accuracy of the identification results of injection attack behavior, in some possible examples, the matching degree recognition subnetwork includes a first matching degree recognition subnetwork; this first matching degree recognition subnetwork is used to identify the matching degree value between background feature data in the image data and sensor data (and acquisition method data) in the first data.

[0147] Correspondingly, such as Figure 7 As shown, if the first image indicates that it does not include the first object, or the first object is in a handheld state, or the first object is a face, step 212 may specifically include: determining the second change situation corresponding to each acquisition time stamp based on the background feature data corresponding to each acquisition time stamp and the background feature data corresponding to the previous acquisition time stamp. For example, in this step, taking acquisition time stamp t_i as an example, specifically, the difference between the background feature data corresponding to acquisition time stamp t_i and the background feature data corresponding to acquisition time stamp t_i-1 can be calculated as the third difference value corresponding to acquisition time stamp t_i. Based on the third difference value corresponding to acquisition time stamp t_i, the second change situation corresponding to acquisition time stamp t_i is determined, wherein the second change situation corresponding to acquisition time stamp t_i includes the third difference value corresponding to acquisition time stamp t_i.

[0148] Accordingly, step 213 can specifically include: based on the first and second changes corresponding to each collection timestamp, obtaining the predicted matching degree value corresponding to each collection timestamp through a first matching degree recognition sub-network. In this step, the first and second changes corresponding to each collection timestamp can be input into the first matching degree recognition sub-network to process the first and second changes corresponding to each collection timestamp, thereby obtaining the predicted matching degree value corresponding to each collection timestamp.

[0149] In some examples, the first matching degree recognition subnetwork includes an encoder and a classifier; or the first matching degree recognition subnetwork includes two encoders (one corresponding to the first change case and the other to the second change case object) and a classifier. The implementation principle of the data processing of this first matching degree recognition subnetwork is similar to that of the aforementioned matching degree recognition subnetwork. Its data processing process can be found in the aforementioned matching degree recognition subnetwork data processing process, which will not be repeated here.

[0150] In some other possible examples, the aforementioned matching degree recognition subnetwork includes a second matching degree recognition subnetwork; this second matching degree recognition subnetwork is used to identify the matching degree value between object feature data in the image data and sensor data (and acquisition method data) in the first data; correspondingly, such as Figure 7 As shown, if the first image indicates that the first object is in a non-handheld state (e.g., placed on a flat surface, such as a card on a table, in which case the first object can be considered to be in a static state, and the movement of the first object shown in the image can be used to determine the movement of the indicated mobile terminal relatively accurately), step 212 may specifically include: determining the second change situation corresponding to each collection time stamp based on the object feature data corresponding to each collection time stamp and the object feature data corresponding to the previous collection time stamp, wherein the object feature data is obtained by identifying the first object in the first image corresponding to each collection time stamp.

[0151] For example, in this step, taking the collection timestamp t_i in each collection timestamp as an example, specifically, the difference between the object feature data corresponding to the collection timestamp t_i and the object feature data corresponding to the collection timestamp t_i-1 can be calculated as the fourth difference value corresponding to the collection timestamp t_i. Based on the fourth difference value corresponding to the collection timestamp t_i, the second change situation corresponding to the collection timestamp t_i is determined, wherein the second change situation corresponding to the collection timestamp t_i includes the fourth difference value corresponding to the collection timestamp t_i.

[0152] Next, step 213 can specifically include: based on the first and second changes corresponding to each collection timestamp, obtaining the predicted matching degree value corresponding to each collection timestamp through a second matching degree recognition sub-network. In this step, the first and second changes corresponding to each collection timestamp can be input into the second matching degree recognition sub-network to process the first and second changes corresponding to each collection timestamp, so as to obtain the predicted matching degree value corresponding to each collection timestamp.

[0153] In some examples, the second matching degree recognition subnetwork includes an encoder and a classifier; or the second matching degree recognition subnetwork includes two encoders (one corresponding to the first change case and the other to the second change case object) and a classifier. The implementation principle of this second matching degree recognition subnetwork in processing data is similar to that of the aforementioned matching degree recognition subnetwork in processing data. Its data processing process can be found in the aforementioned matching degree recognition subnetwork data processing process, and will not be repeated here.

[0154] In some other possible examples, such as Figure 6B and Figure 7 As shown, the injection attack identification network includes a classification subnetwork; correspondingly, step 21 may include steps 31-32:

[0155] In step 31, the first data and image data are input into the classification subnetwork to obtain the predicted classification result.

[0156] For example, the classification subnetwork can be any network that can process time series data. For instance, the classification subnetwork can be a recognition network based on a transformer structure, a recognition network based on a long short-term memory network, a recognition network based on a convolutional neural network, or a recognition network based on an RNN (Recurrent Neural Network).

[0157] In some possible examples, such as Figure 7 As shown, the object feature data from the first data and image data can be input into the classification sub-network to obtain a predictive classification result that indicates whether the image acquisition process involves injection attack behavior. Alternatively, the background feature data from the first data and image data can be input into the classification sub-network to obtain a predictive classification result that indicates whether the image acquisition process involves injection attack behavior. Another approach is to input the object feature data and background feature data from the first data and image data into the classification sub-network to obtain a predictive classification result that indicates whether the image acquisition process involves injection attack behavior.

[0158] The first data may include sensor data and / or acquisition method data.

[0159] In some possible examples, the aforementioned classification subnetwork may include an extraction layer and a classification layer; step 31 may include steps 311-312:

[0160] In step 311, based on the first data and image data, an intermediate representation is obtained through an extraction layer. In this step, the extraction layer can be any extraction layer capable of processing time series data. For example, the extraction layer can be an extraction layer based on a transformer structure, an extraction layer based on a long short-term memory network, an extraction layer based on a convolutional neural network, or an extraction layer based on an RNN.

[0161] In some possible implementations, the mobile terminal can stitch together the first data and the image data to obtain the aforementioned stitched data, and then input the stitched data into the extraction layer to extract features from the stitched data and obtain an intermediate representation.

[0162] For example, the first data may include sensor data (and acquisition method data) corresponding to multiple acquisition timestamps, and the image data may include image data identified from each first image corresponding to multiple acquisition timestamps. The stitched data includes stitched data corresponding to each acquisition timestamp. Taking the acquisition timestamp t_i as an example, the stitching process is explained. Specifically, the sensor data (and acquisition method data) Ci corresponding to the acquisition timestamp t_i in the first data is represented as [q, w, e, r], and the image data Ti corresponding to the acquisition timestamp t_i in the image data is represented as [t, y, u, i]. Correspondingly, the stitched data Ci+Ti corresponding to the acquisition timestamp t_i can be represented as [q, w, e, r, t, y, u, i].

[0163] Next, in step 312, the aforementioned intermediate representation is input into the classification layer to obtain a predictive classification result indicating whether the image acquisition process involves injection attack behavior. In this step, the mobile terminal (the target program) can input the intermediate representation into the classification layer to process the intermediate representation and obtain a predictive classification result indicating whether the image acquisition process involves injection attack behavior.

[0164] For example, this classification layer can be implemented using a multilayer perceptron or other classifiers.

[0165] In some examples, the classification layer can output a probability value indicating that the image acquisition process involves injection attacks. This probability value can characterize the predicted classification result of whether the image acquisition process involves injection attacks. For example, if the probability value is greater than or equal to a preset probability threshold, the predicted classification result can be considered to indicate that the image acquisition process involves injection attacks; if the probability value is less than the preset probability threshold, the predicted classification result can be considered to indicate that the image acquisition process involves injection attacks.

[0166] After obtaining the predicted classification result, in step 32, based on the predicted classification result, it is determined whether the image acquisition process involves injection attack behavior. In some possible examples, if the predicted classification result indicates that the image acquisition process involves injection attack behavior, then it is determined that the image acquisition process involves injection attack behavior; if the predicted classification result indicates that the image acquisition process does not involve injection attack behavior, then it is determined that the image acquisition process does not involve injection attack behavior.

[0167] In some possible examples, such as Figure 7 As shown, the injection attack identification network includes the aforementioned matching identification subnetwork and classification subnetwork (and the time coding subnetwork corresponding to the subsequent classification subnetwork). It combines the predicted matching degree values ​​obtained through the matching identification subnetwork with the predicted classification results obtained through the classification subnetwork to jointly determine whether the image acquisition process involves injection attack behavior. For example, to better protect user privacy and property security, if at least one result from the predicted matching degree values ​​obtained through the matching identification subnetwork and the predicted classification results obtained through the classification subnetwork indicates that the image acquisition process involves injection attack behavior, then the final result is determined to be that the image acquisition process involves injection attack behavior. Alternatively, it can be set such that if at least one result from the predicted matching degree values ​​obtained through the matching identification subnetwork and the predicted classification results obtained through the classification subnetwork indicates that the image acquisition process does not involve injection attack behavior, then the final result is determined to be that the image acquisition process does not involve injection attack behavior.

[0168] In some possible examples, the aforementioned injection attack identification network includes a time-coding subnetwork and a classification subnetwork; correspondingly, step 21 may include steps 41-43:

[0169] In step 41, each collection timestamp is input into the time coding sub-network to obtain the time coding data corresponding to each collection timestamp.

[0170] The time-coding subnetwork encodes each acquisition timestamp, converting them into vector representations that capture the complexity of the time dimension. Specifically, the mobile terminal inputs each acquisition timestamp into the time-coding subnetwork, which processes the timestamps to obtain the time-coded data corresponding to each timestamp. For example, this time-coding subnetwork can be implemented using the Time2vec algorithm. The Time2vec algorithm is a special encoding method that captures the periodicity and patterns of time-series data. It encodes timestamps to capture the complexity of the time dimension, obtaining time-coded data (i.e., vector representations) that characterize this complexity. Specifically, the time2vec algorithm can be expressed by the following formula: ; (1)

[0171] in, Represents each collection timestamp The corresponding time-coded data; Indicates the collection timestamp The kth element in the corresponding time-coded data, and These are trainable network parameters; It is a set periodic activation function, which can be set as a sine function, for example... It can also be set to a cosine function; These are the preset hyperparameters.

[0172] After the mobile terminal obtains the first data and image data, it inputs each acquisition timestamp into the time coding sub-network, so that the time coding sub-network can obtain the time-coded data corresponding to each acquisition timestamp through the above formula (1). It can be understood that the time-coded data corresponding to a single acquisition timestamp is one... A dimensional vector.

[0173] In other examples, the time-coding subnetwork can also be implemented using other algorithms that can encode timestamp data, such as sine-based encoding algorithms and cosine-based encoding algorithms.

[0174] In step 42, the first data, image data, and time-coded data corresponding to each acquisition timestamp are input into the classification sub-network to obtain the predicted classification result.

[0175] For example, the classification subnetwork includes an extraction layer and a classification layer. The extraction layer can be an extraction layer based on a transformer structure, an extraction layer based on a long short-term memory network, an extraction layer based on a convolutional neural network, or an extraction layer based on an RNN. In this step, the mobile terminal (the target program) combines the sensor data (and acquisition method data) corresponding to each acquisition timestamp in the first data, the image data corresponding to each acquisition timestamp in the image data, and the time-coded data corresponding to each acquisition timestamp to obtain an intermediate data sequence. For example, for any acquisition timestamp j in the first data, assuming its corresponding sensor data (and acquisition method data) is represented as x1j (where x1j can be an array), its corresponding image data is represented as x2j, and its time-coded data is represented as x3j, combining the sensor data (and acquisition method data), image data, and time-coded data corresponding to acquisition timestamp j yields the intermediate data corresponding to acquisition timestamp j, which can be represented as [x1j, x2j, x3j]. By doing so, the intermediate data corresponding to each collection timestamp is obtained, and thus an intermediate data sequence is formed.

[0176] The intermediate data sequence is then input into the extraction layer of the classification subnetwork to process the intermediate data sequence and obtain intermediate representations.

[0177] In some possible implementations, the extraction layer of the classification subnetwork can be an extraction layer based on a transformer structure; this extraction layer is used to perform convolution processing on the input and processing based on a self-attention mechanism; correspondingly, the aforementioned process of obtaining intermediate representations can include: inputting an intermediate data sequence into the extraction layer, so that the extraction layer performs convolution processing on the intermediate data sequence to obtain convolution processing result data, and processing the convolution processing result data based on a self-attention mechanism to obtain intermediate representations.

[0178] In the above steps, the extraction layer performs convolution processing on the intermediate data sequence, which can realize the feature extraction of local contextual information in the intermediate data sequence. Then, based on the self-attention mechanism, the convolution processing result data is processed to realize the extraction of the overall relationship of the data, and obtain intermediate representations that are more helpful to improve the classification results. This can better enhance the ability of the injection classification sub-network to model local contextual information in time series data, thereby reducing the impact of abnormal data in time series data (i.e., intermediate data sequence) on subsequent classification results and improving the accuracy of classification results. That is, it can achieve the accuracy of identifying whether the image acquisition process involves injection attack behavior.

[0179] Next, the aforementioned intermediate representation is input into the classification layer to obtain a predicted classification result indicating whether the image acquisition process involves injection attack behavior. In this step, the aforementioned intermediate representation is input into the classification layer so that the classification layer processes the intermediate representation to obtain a predicted classification result indicating whether the image acquisition process involves injection attack behavior.

[0180] Next, in step 43, based on the predicted classification results, it is determined whether the image acquisition process involves injection attacks. The implementation principle of this step is similar to that of step 32 mentioned above, and its implementation process can be found in the implementation process of step 32, which will not be repeated here.

[0181] In the above process, by encoding time information (i.e., each collection timestamp) into the intermediate representation through the time coding subnetwork and the classification subnetwork, the classification subnetwork can better understand and process time series data, i.e. first data and image data, and obtain more accurate classification results based on the intermediate representation that contains time information and the feature information corresponding to each time information (i.e., sensor data, collection method data and image data corresponding to each collection timestamp).

[0182] The above embodiments provide a new method for identifying injection attack behavior. Considering that in a real image acquisition process (i.e., an image acquisition process that does not involve injection attack behavior), the movement of the mobile terminal during the image acquisition process (and changes in ambient light) is related to the movement of the first object (and changes in image brightness) in the image sequence acquired during the image acquisition process. For example, if the mobile terminal moves to the right relative to the first object, the corresponding image sequence will indicate that the first object (and / or the background in the image) moves to the left relative to the mobile terminal. Or, if the mobile terminal moves closer to the first object, the corresponding image sequence will indicate that the first object (and / or the background in the image) becomes larger. In other words, the image sequence can also indicate the movement of the mobile terminal. Or, if the ambient light in the environment where the mobile terminal is located becomes weaker, the corresponding image brightness value in the image sequence will become smaller. Therefore, in the embodiments provided in this specification, by identifying and determining the correlation between the first data corresponding to the image acquisition process, namely sensor data (which can characterize the motion of the mobile terminal and / or changes in ambient light during the image acquisition process), and the image data corresponding to the image acquisition process (which can also indicate the motion of the mobile terminal and / or changes in ambient light), for example, determining whether the motion of the mobile terminal indicated by the two is consistent (and determining whether the changes in ambient light indicated by the two are consistent), it is determined whether the image acquisition process involves injection attack behavior, thereby realizing the identification of injection attack behavior.

[0183] Furthermore, considering the significant difference in mobile terminal movement between image acquisition processes involving injection attacks and genuine image acquisition processes (i.e., image acquisition processes without injection attacks), specifically, in genuine image acquisition, the mobile terminal generally moves in sync with the user's acquisition actions (i.e., the user controls the mobile terminal's acquisition actions according to image acquisition instructions). Consequently, in genuine image acquisition, the mobile terminal's movement and the user's acquisition actions are consistently correlated. However, in image acquisition processes involving injection attacks, the mobile terminal is generally flat and / or stationary, resulting in insufficient consistency between the mobile terminal's movement and the user's acquisition actions. Therefore, the first data corresponding to the image acquisition process also includes acquisition method data to instruct the user to use a specific acquisition method. This data, combined with the correlation between sensor data, acquisition method data, and image data, helps identify whether the image acquisition process involves injection attacks, thereby better enabling the identification of injection attacks.

[0184] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0185] Corresponding to the above method embodiments, this specification provides an embodiment of an injection attack behavior identification device 800, the schematic block diagram of which is shown below. Figure 8 As shown, it includes: a first acquisition module 810, configured to acquire first data corresponding to the image acquisition process of a mobile terminal for a first object, wherein the first data includes sensor data, the sensor data corresponding to sensors in the mobile terminal, and the sensors including a first sensor for sensing the motion of the mobile terminal; a second acquisition module 820, configured to acquire image data corresponding to the image acquisition process, wherein the image data is obtained from a first image recognition corresponding to the image acquisition process, and the image data is used to indicate the motion of the mobile terminal; and a first determination module 830, configured to use the first data and the image data to determine whether the image acquisition process involves injection attack behavior.

[0186] In some possible examples, the first object is a card or a face.

[0187] In some possible examples, the first data may also include acquisition method data, which is used to instruct the user to use a specific acquisition method during the image acquisition process.

[0188] In some possible examples, the acquisition method data includes: data indicating a frontal acquisition method and / or data indicating a non-frontal acquisition method, wherein the non-frontal acquisition method includes: acquisition method with random multiple tilt angles.

[0189] In some possible examples, it also includes: a generation module (not shown in the figure) configured to randomly generate a first instruction during the image acquisition process, the first instruction being used to instruct a preset vibrator in the mobile terminal to vibrate; the first data also includes: data indicating whether the mobile terminal vibrates.

[0190] In some possible examples, the mobile terminal is equipped with multiple types of sensors; when the first data is the actual data corresponding to the image acquisition process, the sensor data includes data directly measured by the multiple types of sensors and / or the pose data of the mobile terminal determined based on the data measured by the first sensor.

[0191] In some possible examples, the first data includes sensor data corresponding to multiple acquisition timestamps; the image data includes image data identified from first images corresponding to multiple acquisition timestamps.

[0192] In some possible examples, the second acquisition module 820 is specifically configured to, when the first image is detected to contain the first object, detect the object feature data corresponding to the first object from the first image and classify it into the image data corresponding to the first image.

[0193] In some possible examples, the object feature data includes at least one of the following types of data: regional location information of the image region where the first object is located in the first image, Euler angles, sharpness value and brightness value corresponding to the first object, and a first estimated distance between the first object and the mobile terminal, wherein the first estimated distance is determined based on the proportion of the image region where the first object is located in the first image, and the Euler angles, sharpness value and brightness value corresponding to the first object are determined based on the first image and a pre-trained attribute recognition network.

[0194] In some possible examples, the second acquisition module 820 is also configured as follows:

[0195] If the first image does not contain the first object, background feature data is extracted from the first image and incorporated into the image data corresponding to the first image. The background feature data is used to calculate the motion of the first image relative to its previous frame.

[0196] In some possible examples, the second acquisition module 820 is also configured as follows:

[0197] If the first image is detected to contain the first object, the image region containing the first object is removed from the first image to obtain the background region image corresponding to the first image;

[0198] Background feature data is extracted from the background region image and incorporated into the image data corresponding to the first image. The background feature data is used to calculate the motion of the first image relative to its previous frame image.

[0199] In some possible examples, the first determining module 830 is specifically configured to use the first data and the image data to determine whether the image acquisition process involves injection attack behavior through an injection attack identification network. The injection attack identification network is a network pre-trained based on the sample data and corresponding label data of each sample acquisition process. The sample data includes the sample sensor data and sample image data corresponding to the corresponding sample acquisition process, and the label data is used to indicate whether the corresponding sample acquisition process involves injection attack behavior.

[0200] In some possible examples, the first data includes sensor data corresponding to multiple acquisition timestamps, and the image data includes image data identified from each first image corresponding to multiple acquisition timestamps; the injection attack identification network includes a matching degree identification subnetwork.

[0201] The first determining module 830 includes: a first determining unit (not shown in the figure), configured to determine the first change situation corresponding to each collection time stamp based on the sensor data corresponding to each collection time stamp and the sensor data corresponding to the previous collection time stamp;

[0202] The second determining unit (not shown in the figure) is configured to determine the second change situation corresponding to each acquisition time stamp based on the image data corresponding to each acquisition time stamp and the image data corresponding to the previous acquisition time stamp.

[0203] The first obtaining unit (not shown in the figure) is configured to obtain the predicted matching degree value between the sensor data and image data corresponding to each acquisition time stamp based on the first change and the second change corresponding to each acquisition time stamp through the matching degree recognition sub-network.

[0204] The third determining unit (not shown in the figure) is configured to determine whether the image acquisition process involves injection attack behavior based on each predicted matching degree value.

[0205] In some possible examples, the matching degree recognition subnetwork includes a first matching degree recognition subnetwork;

[0206] If the first image indicates that it does not include the first object or the first object is in a handheld state, the second determining unit is specifically configured to determine the second change situation corresponding to each collection time stamp based on the background feature data corresponding to each collection time stamp and the background feature data corresponding to the previous collection time stamp.

[0207] The first obtaining unit is specifically configured to obtain the predicted matching degree value corresponding to each collection time stamp based on the first change and the second change corresponding to each collection time stamp through the first matching degree recognition sub-network.

[0208] In some possible examples, the matching degree recognition subnetwork includes a second matching degree recognition subnetwork;

[0209] If the first image indicates that the first object is not in a handheld state, the second determining unit is specifically configured to determine the second change situation corresponding to each collection time stamp based on the object feature data corresponding to each collection time stamp and the object feature data corresponding to the previous collection time stamp, wherein the object feature data is identified based on the first object in the first image corresponding to each collection time stamp;

[0210] The first obtaining unit is specifically configured to obtain the predicted matching degree value corresponding to each collection time stamp through the second matching degree recognition sub-network based on the first change and the second change corresponding to each collection time stamp.

[0211] In some possible examples, the injection attack identification network includes a classification subnetwork;

[0212] The first determining module 830 includes: a second obtaining unit (not shown in the figure), configured to input the first data and the image data into the classification sub-network to obtain a predicted classification result;

[0213] The fourth determining unit (not shown in the figure) is configured to determine whether the image acquisition process involves injection attack behavior based on the predicted classification result.

[0214] In some possible examples, the injection attack detection network also includes a time-coded subnetwork:

[0215] The first determining module 830 further includes: a third obtaining unit (not shown in the figure), configured to input each collection timestamp into the time coding subnetwork to obtain time coding data corresponding to each collection timestamp before inputting the first data and the image data into the classification subnetwork to obtain the predicted classification result;

[0216] The second obtaining unit is configured to input the first data, the image data, and the time-coded data corresponding to each acquisition timestamp into the classification sub-network to obtain the predicted classification result.

[0217] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0218] This specification also provides a computer-readable storage medium storing a computer program that, when executed in a computer, causes the computer to perform the injection attack identification method provided in this specification.

[0219] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the injection attack behavior identification method provided in this specification.

[0220] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0221] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of the present invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0222] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying injection attack behavior, comprising: Acquire first data corresponding to the image acquisition process of the mobile terminal for the first object, wherein the first data includes sensor data, the sensor data corresponds to the sensors in the mobile terminal, and the sensors include a first sensor for sensing the motion of the mobile terminal; The image data corresponding to the image acquisition process is obtained, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal; the acquisition of the image data corresponding to the image acquisition process includes: when the first image is detected to contain the first object, removing the image region where the first object is located from the first image to obtain a background region image corresponding to the first image; extracting background feature data from the background region image and incorporating it into the image data corresponding to the first image, wherein the background feature data is used to calculate the movement of the first image relative to its previous frame image; By using the motion of the mobile terminal represented by the first data and the motion of the mobile terminal indicated by the image data, it is determined whether the image acquisition process involves injection attack behavior.

2. The method as described in claim 1, wherein, The first object is a card or a face.

3. The method as described in claim 1, wherein, The first data also includes acquisition method data, which is used to instruct the user to use a specific acquisition method during the image acquisition process.

4. The method of claim 3, wherein, The data acquisition method includes: data indicating frontal acquisition method and / or data indicating non-frontal acquisition method, wherein the non-frontal acquisition method includes: acquisition method with random multiple tilt angles.

5. The method of claim 1, further comprising: During the image acquisition process, a first instruction is randomly generated, which is used to instruct a preset vibrator in the mobile terminal to vibrate. The first data also includes data indicating whether the mobile terminal is vibrating.

6. The method of claim 1, wherein, The mobile terminal is equipped with multiple types of sensors; When the first data is the actual data corresponding to the image acquisition process, the sensor data includes data directly measured by the plurality of types of sensors and / or the pose data of the mobile terminal determined based on the data measured by the first sensor.

7. The method of claim 1, wherein, The first data includes sensor data corresponding to multiple acquisition timestamps during the image acquisition process; the image data includes image data identified from the first images corresponding to the multiple acquisition timestamps.

8. The method of claim 1, wherein, The step of acquiring the image data corresponding to the image acquisition process also includes: If the first image is detected to contain the first object, the object feature data corresponding to the first object is detected from the first image and included in the image data corresponding to the first image.

9. The method of claim 8, wherein, The object feature data includes at least one of the following types of data: regional location information of the image region where the first object is located in the first image, Euler angles, sharpness value and brightness value corresponding to the first object, and a first estimated distance between the first object and the mobile terminal, wherein the first estimated distance is determined based on the proportion of the image region where the first object is located in the first image, and the Euler angles, sharpness value and brightness value corresponding to the first object are determined based on the first image and a pre-trained attribute recognition network.

10. The method of claim 1, further comprising: If the first image does not contain the first object, background feature data is extracted from the first image and incorporated into the image data corresponding to the first image. The background feature data is used to calculate the motion of the first image relative to its previous frame.

11. The method according to any one of claims 1-10, wherein, Determining whether the image acquisition process involves injection attacks includes: Using the first data and the image data, an injection attack identification network is used to determine whether the image acquisition process involves injection attack behavior. The injection attack identification network is a network pre-trained based on the sample data and corresponding label data of each sample acquisition process. The sample data includes the sample sensor data and sample image data corresponding to the corresponding sample acquisition process, and the label data is used to indicate whether the corresponding sample acquisition process involves injection attack behavior.

12. The method of claim 11, wherein, The first data includes sensor data corresponding to multiple acquisition timestamps, and the image data includes image data identified from each first image corresponding to multiple acquisition timestamps; The injection attack identification network includes a matching degree identification sub-network; Determining whether the image acquisition process involves injection attacks includes: Based on the sensor data corresponding to each collection timestamp and the sensor data corresponding to the previous collection timestamp, determine the first change situation corresponding to each collection timestamp. Based on the image data corresponding to each collection timestamp and the image data corresponding to the previous collection timestamp, determine the second change situation corresponding to each collection timestamp. Based on the first and second changes corresponding to each acquisition time stamp, the predicted matching degree value between the sensor data and image data corresponding to each acquisition time stamp is obtained through the matching degree recognition sub-network. Based on each predicted matching degree value, it is determined whether the image acquisition process involves injection attack behavior.

13. The method of claim 12, wherein, The matching degree recognition subnetwork includes a first matching degree recognition subnetwork; If the first image indicates that it does not include the first object or the first object is in a handheld state, the step of determining the second change corresponding to each acquisition time stamp based on the image data corresponding to each acquisition time stamp and the image data corresponding to the previous acquisition time stamp includes: Based on the background feature data corresponding to each collection timestamp and the background feature data corresponding to the previous collection timestamp, the second change situation corresponding to each collection timestamp is determined. The step of obtaining the predicted matching degree value between the sensor data and image data corresponding to each acquisition timestamp includes: Based on the first and second changes corresponding to each collection timestamp, the predicted matching degree value corresponding to each collection timestamp is obtained through the first matching degree recognition sub-network.

14. The method of claim 12, wherein, The matching degree recognition subnetwork includes a second matching degree recognition subnetwork; If the first image indicates that the first object is not in a handheld state, the step of determining the second change corresponding to each acquisition time stamp based on the image data corresponding to each acquisition time stamp and the image data corresponding to the previous acquisition time stamp includes: Based on the object feature data corresponding to each collection time stamp and the object feature data corresponding to the previous collection time stamp, the second change situation corresponding to each collection time stamp is determined, wherein the object feature data is identified based on the first object in the first image corresponding to each collection time stamp; The step of obtaining the predicted matching degree value between the sensor data and image data corresponding to each acquisition timestamp includes: Based on the first and second changes corresponding to each collection timestamp, the predicted matching degree value corresponding to each collection timestamp is obtained through the second matching degree recognition sub-network.

15. The method of claim 11, wherein, The injection attack identification network includes a classification subnetwork; Determining whether the image acquisition process involves injection attacks includes: The first data and the image data are input into the classification sub-network to obtain the predicted classification result; Based on the predicted classification results, it is determined whether the image acquisition process involves injection attacks.

16. The method of claim 15, wherein the injection attack identification network further comprises a time-coded sub-network: Before inputting the first data and the image data into the classification subnetwork to obtain the predicted classification result, the method further includes: Input each collection timestamp into the time coding sub-network to obtain the time coding data corresponding to each collection timestamp; The obtained predicted classification result includes: The first data, the image data, and the time-coded data corresponding to each acquisition timestamp are input into the classification sub-network to obtain the predicted classification result.

17. A device for identifying injection attack behavior, comprising: The first acquisition module is configured to acquire first data corresponding to the image acquisition process of the mobile terminal for the first object, wherein the first data includes sensor data, the sensor data corresponds to the sensors in the mobile terminal, and the sensors include a first sensor for sensing the motion of the mobile terminal; The second acquisition module is configured to acquire image data corresponding to the image acquisition process, wherein the image data is obtained from the first image recognition corresponding to the image acquisition process, and the image data is used to indicate the movement of the mobile terminal. Specifically, the second acquisition module is configured to, when the first image is detected to contain the first object, remove the image region where the first object is located from the first image to obtain a background region image corresponding to the first image; extract background feature data from the background region image and classify it into the image data corresponding to the first image, wherein the background feature data is used to calculate the movement of the first image relative to its previous frame image. The first determining module is configured to determine whether the image acquisition process involves injection attack behavior by using the motion of the mobile terminal represented by the first data and the motion of the mobile terminal indicated by the image data.

18. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-16.

Images