Data processing method, service providing method, device, equipment and storage medium

Abstract

An embodiment of the present invention provides a data processing method, a service provision method, an apparatus, a device, and a storage medium. The data processing method includes: a processing device obtains an image sequence to be detected containing a target object, and simultaneously extracts information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected. Then, the depth value of the image feature is determined to determine the type of the target object based on the depth value. In the above method, the information of the image sequence to be detected in the time dimension and the space dimension will be completely retained in the image feature. Then, the type of the target object is identified based on this information-rich image feature, so that the type of the target object can be accurately determined, ensuring that the processing device will not be attacked by a specific type of target object.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, a service providing method, an apparatus, a device, and a storage medium. Background Art

[0002] Facial recognition is increasingly being used in daily life, for example, in attendance statistics and personnel management. However, facial recognition is often susceptible to non-liveness attacks. For example, in attendance statistics, a non-liveness attack can be launched against the facial recognition system using employee photos or videos. This attack tricks the facial recognition system into recognizing an employee's photo or video, thereby impersonating the employee and clocking in.

[0003] The above-mentioned non-liveness attack will bring hidden dangers to the security of the face recognition system. Therefore, how to prevent the recognition system from being attacked by non-liveness has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data method, a service providing method, an apparatus, a device, and a storage medium to ensure the accuracy of target object classification, thereby avoiding attacks from non-living objects.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0006] Obtaining a sequence of images to be detected;

[0007] Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0008] determining a depth value of the image feature;

[0009] The type of the target object contained in the to-be-detected image sequence is determined according to the depth value.

[0010] In a second aspect, an embodiment of the present invention provides a data processing device, including:

[0011] An acquisition module, used for acquiring an image sequence to be detected;

[0012] An extraction module, configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0013] A depth value determination module, configured to determine a depth value of the image feature;

[0014] The type determination module is configured to determine the type of the target object contained in the image sequence to be detected according to the depth value.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the data processing method described in the first aspect. The electronic device may also include a communication interface for communicating with other devices or a communication network.

[0016] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the data processing method described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present invention provides a service providing method, including:

[0018] In response to an image acquisition operation triggered by a user, acquiring a sequence of images to be detected;

[0019] Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0020] Output a depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected.

[0021] In a sixth aspect, an embodiment of the present invention provides a service providing device, including:

[0022] An acquisition module, configured to acquire a sequence of images to be detected in response to an image acquisition operation triggered by a user;

[0023] An extraction module, configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0024] The output module is configured to output a depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected.

[0025] In a seventh aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the service provision method described in the fifth aspect. The electronic device may also include a communication interface for communicating with other devices or a communication network.

[0026] In the eighth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the service providing method described in the fifth aspect.

[0027] In a ninth aspect, an embodiment of the present invention provides a service providing method, including:

[0028] Receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence;

[0029] Inputting the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in a time dimension and a space dimension to obtain a first image feature;

[0030] Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature;

[0031] Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature;

[0032] Inputting the feature fusion result into the depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence;

[0033] Adjusting model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence;

[0034] The model parameters are output.

[0035] In a tenth aspect, an embodiment of the present invention provides a service providing device, including:

[0036] A receiving module, configured to receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence;

[0037] an input module, configured to input the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in a temporal dimension and a spatial dimension to obtain a first image feature;

[0038] Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature;

[0039] Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature;

[0040] Inputting the feature fusion result into the depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence;

[0041] An adjustment module, configured to adjust model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence;

[0042] An output module is used to output the model parameters.

[0043] In an eleventh aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the service provision method described in the ninth aspect. The electronic device may further comprise a communication interface for communicating with other devices or a communication network.

[0044] In the twelfth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the service providing method described in the ninth aspect.

[0045] In the data processing method provided by an embodiment of the present invention, a processing device obtains an image sequence to be detected containing a target object, and simultaneously extracts information of the image sequence to be detected in the time dimension and the space dimension to obtain image features. The depth value of the image feature is then identified to determine the type of the target object based on the depth value. In other words, the information of the image sequence to be detected in the time dimension and the space dimension will be completely retained in the image feature, and there will be no large amount of information loss. The type of the target object is then identified based on this information-rich image feature, so that the type of the target object can be accurately determined, further ensuring that the processing device with face recognition function will not be attacked by a specific type of target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;

[0048] Figure 2 A flowchart of another data processing method provided by an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the structure of a preset model provided by an embodiment of the present invention;

[0050] Figure 4 A flowchart of an image feature extraction method provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of the structure of another preset model provided in an embodiment of the present invention;

[0052] Figure 6 A schematic structural diagram of another preset model provided in an embodiment of the present invention;

[0053] Figure 7 A schematic structural diagram of another preset model provided in an embodiment of the present invention;

[0054] Figure 8 A flowchart of another data processing method provided by an embodiment of the present invention;

[0055] Figure 9 A flow chart of a service providing method provided in an embodiment of the present invention;

[0056] Figure 10 A flowchart of another service providing method provided by an embodiment of the present invention;

[0057] Figure 11 A schematic diagram of the data processing method provided by an embodiment of the present invention applied in a clock-in / clock-out scenario;

[0058] Figure 12 A schematic diagram of the data processing method provided by an embodiment of the present invention applied in a retail scenario;

[0059] Figure 13 A schematic structural diagram of a data processing device provided by an embodiment of the present invention;

[0060] Figure 14 For Figure 13 A schematic structural diagram of an electronic device corresponding to the data processing device provided in the illustrated embodiment;

[0061] Figure 15 A schematic structural diagram of a service providing device provided in an embodiment of the present invention;

[0062] Figure 16 For Figure 15 A schematic structural diagram of an electronic device corresponding to the service providing apparatus provided in the illustrated embodiment;

[0063] Figure 17 A schematic structural diagram of another service providing device provided in an embodiment of the present invention;

[0064] Figure 18 For Figure 17 A schematic structural diagram of an electronic device corresponding to the service providing apparatus provided in the illustrated embodiment. DETAILED DESCRIPTION

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. "A plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0067] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0068] As used herein, the words “if” and “if” may be interpreted as “at the time of” or “when” or “in response to determining” or “in response to identifying,” depending on the context. Similarly, the phrases “if it is determined” or “if (stated condition or event) is identified” may be interpreted as “when it is determined” or “in response to determining” or “when identifying (stated condition or event)” or “in response to identifying (stated condition or event),” depending on the context.

[0069] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0070] Before describing in detail the data processing method provided by the embodiment of the present invention, the practical significance of the data processing method may be described as follows:

[0071] For a processing device with facial recognition capabilities, in the attendance clocking scenario mentioned in the background technology, employee B can use a photo or video of employee A to enable the processing device to obtain an image sequence corresponding to the non-living employee A, where the image sequence contains employee A's face, which is the target object of the image sequence. When the processing device further determines through its own facial recognition function that the target object contained in the image sequence is employee A, it will be considered that employee A has completed the attendance clocking in, but this clocking in is clearly an act of cheating. In this case, the data processing method provided by the present invention can accurately identify whether the employee contained in the image sequence is alive, and further determine whether the employee's attendance clocking in is valid, thereby avoiding the above situation.

[0072] Similarly, employees in a company may have different access permissions for different work areas. For example, employee A has access to work area 1, while employee B does not. In practice, employee B could use a photo or video of employee A to attack the facial recognition system in the processing device, thereby gaining access to work area 1. However, this is clearly illegal. In this case, the data processing method provided by the present invention can be used to avoid this situation.

[0073] Of course, in addition to the above scenarios, the method provided by the present invention can be used in any scenario where user identification is performed through face recognition. For a wider range of applications, the method provided by the present invention can be used in any scenario where there is a need to identify whether the target object contained in an image sequence is a living person, and the target object is not limited to a person.

[0074] Based on the above description, some embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the absence of conflicts between the embodiments, the following embodiments and the features in the embodiments may be combined with each other. In addition, the step timings in the following method embodiments are only examples and are not strictly limiting.

[0075] The following embodiments are described by taking the scenario of identifying whether the face of a person included in an image sequence is alive as an example. Figure 1 This is a flow chart of a data processing method provided by an embodiment of the present invention. The data processing method provided by an embodiment of the present invention can be executed by a processing device with a face recognition function. It is understandable that the processing device can be implemented as software, or a combination of software and hardware. Figure 1 As shown, the method includes the following steps:

[0076] S101: Acquire a sequence of images to be detected.

[0077] To acquire the image sequence to be detected, the processing device can optionally be configured with a camera. In response to a user-triggered permission verification operation on the processing device, the processing device can control the camera to capture an image sequence, i.e., the image sequence to be detected. Alternatively, the user can trigger a permission verification operation on a terminal device, which then captures the image sequence to be detected. The terminal device then transmits the image sequence to the processing device.

[0078] The image sequence to be detected is actually a short video clip, such as 2 seconds. Furthermore, the image sequence to be detected may contain a target object, most commonly a human face. The image sequence to be detected captured by the camera may correspond to a living human face, or it may correspond to a non-living human face, such as a photograph or video containing a human face. In this case, the following steps are required to determine the type of target object in the image sequence, that is, whether it is a living person.

[0079] Optionally, the user can trigger the permission verification operation by clicking or using voice commands, etc. In addition, the permissions that need to be verified may be diverse for different usage scenarios, such as the permission to punch in or the permission to use the work area.

[0080] S102 , extracting information of the image sequence to be detected in the time dimension and the space dimension simultaneously to obtain image features of the image sequence to be detected.

[0081] Next, the processing device can extract features from the collected image sequence to be detected as a whole, thereby simultaneously extracting information about the image sequence to be detected in both the temporal and spatial dimensions. For the sake of simplicity in the subsequent description, in this embodiment, the image features of the image sequence to be detected may be referred to as target image features.

[0082] After feature extraction, the temporal and spatial information of the image sequence to be detected is relatively intact within the target image features. Compared to first extracting spatial information from the image sequence to be detected and then extracting temporal information from the extracted results, this simultaneous extraction method in this embodiment avoids significant information loss. An alternative method for simultaneously extracting multi-dimensional information is three-dimensional convolution, although other methods are also possible.

[0083] S103: Determine the depth value of the image feature.

[0084] S104: Determine the type of the target object contained in the image sequence to be detected according to the depth value.

[0085] Depth estimation is then performed on this target image feature containing multi-dimensional information to obtain a depth value for the target image feature. The type of the target object contained in the image to be detected is then determined based on the estimated depth value. The type of the target object can reflect whether the target object is living. An optional type determination method is to determine that the target object is living when the depth value meets a preset condition; otherwise, the target object is determined to be non-living. This type determination can determine whether the processing device is being attacked by a non-living target object.

[0086] In practical applications, the target image feature obtained in step 102 can be specifically expressed as an N*N matrix. After depth value estimation, the depth value obtained is actually also a matrix, that is, a depth value matrix of size N*N. In other words, each element in the target image feature has a unique corresponding depth value. Based on the values ​​of each element in the depth value matrix, the above-mentioned preset condition can be set as follows: the range of variation between the values ​​of each element in the depth value matrix is ​​greater than or equal to the preset range, specifically, the difference between the maximum and minimum values ​​is greater than or equal to the preset range. The preset range can be a smaller range, such as 0 to 0.5, etc.

[0087] The reason these pre-set conditions can be used to determine whether a target object is alive is that a living human face is three-dimensional, meaning that the facial features have varying degrees of prominence. This results in significant variations in the values ​​of the elements in the depth matrix, i.e., a wide range of variation. For example, the nose is the most prominent feature, so the depth value corresponding to it in the depth matrix is ​​the smallest, while the depth values ​​corresponding to other parts of the face are larger.

[0088] In contrast, a non-living human face is planar, meaning the facial features all have the same degree of protrusion, resulting in nearly identical values ​​for each element in the depth matrix. However, given the error in depth estimation, the differences between the values ​​of each element in the depth matrix are relatively small, meaning the range of variation is relatively small.

[0089] In this embodiment, the processing device obtains a sequence of images to be detected that contains a target object, and extracts features from the sequence of images to be detected by means of three-dimensional convolution to obtain image features. The depth value of the image feature is then identified, and the type of the target object is determined based on the depth value. Since the information of the image sequence to be detected in both the time dimension and the space dimension can be extracted simultaneously through three-dimensional convolution, that is, the information in the image sequence to be detected in both the time dimension and the space dimension will be retained in the image features, and there will be no large amount of information loss. Therefore, the type of the target object is identified based on this information-rich image feature, thereby being able to accurately determine the type of the target object, and further ensuring that the processing device will not be attacked by a specific type of non-living target object.

[0090] Optionally, if the target object is determined to be a non-living type in the above manner, indicating that an attack on the processing device has occurred using a photo or video, the processing device can optionally directly ignore the permission verification operation triggered by the user, or display a prompt message to the user indicating that the permission verification failed.

[0091] If the target object is determined to be a living entity, the target object can be considered the user who triggered the permission verification operation. The processing device can then further identify the target object's identification information. This identification information can be considered the identity information of the user who triggered the permission verification operation. Based on the preset correspondence between the identification information and the permission, it is determined whether the permission verification operation triggered by the user is valid. Based on the validity of the permission verification operation, the corresponding response action is executed. As will be readily understood, different permissions correspond to different response actions.

[0092] Taking the attendance punching scenario as an example, in this case, the permission operation is the attendance punching operation. If the target object is not alive, a prompt message indicating that the punching failed is directly displayed. If the target object is alive, the identification information of the target object can be further determined, and based on this identification information, it can be determined whether the target object is an employee of the company and whether the permission to trigger the punching operation can be determined. If the target object is an employee of the company, it naturally has the permission to trigger the punching operation, that is, the attendance punching operation triggered by the user is valid, and it is determined that the target object has successfully punched in. In this scenario, the above-mentioned processing device can specifically be an attendance machine.

[0093] Taking the work area usage scenario as an example, in this case, the permission operation is the access control permission verification operation. If the target object is non-living, the prompt information directly displayed to the user who triggered the permission verification operation is: no permission to use work area 1. If the target object is alive, it can be determined based on the target object's identification information whether the target object is an employee of the company and whether it has permission to use work area 1. If the target object has permission to use work area 1, that is, the access control permission verification operation triggered by the user is valid, the processing device can control the work area 1 to open the door to allow the target object to enter. In this scenario, the above-mentioned processing device can specifically be an access control system.

[0094] Optionally, step 102 in the above embodiment can be implemented with the aid of a preset model. Specifically, the image sequence to be detected is input into the preset model, and the preset model extracts image features of the image sequence to be detected. The image feature extraction is specifically implemented by a feature extraction network in the preset model. Similarly, steps 103 and 104 in the above embodiment can also be implemented with the aid of a preset model. The preset model can be a model based on a convolutional neural network.

[0095] but Figure 2 Flowchart of another data processing method provided by an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0096] S201: Acquire a sequence of images to be detected.

[0097] The execution process of the above step 201 is similar to the corresponding steps of the above embodiment, which can be seen in FIG. Figure 1 The relevant descriptions in the illustrated embodiments are not repeated here.

[0098] S202: Input the image sequence to be detected into a feature extraction network in a preset model, so that the preset model extracts image features of the image sequence to be detected.

[0099] S203: Input the image features of the image sequence to be detected into a depth estimation network in a preset model, so that the depth estimation network estimates the depth value of the image features.

[0100] S204: Input the depth value into a classification network in a preset model so that the classification network determines the type of the target object.

[0101] After obtaining the image sequence to be detected, it can be input into the prediction model so that the prediction model can extract features of the image sequence to be detected, that is, extract information of the image sequence to be detected in the time dimension and the space dimension at the same time to obtain the image features of the image sequence to be detected. Then, the image features of the image sequence to be detected are input into the depth estimation network in the preset model so that the depth estimation network estimates the depth value of the image feature. The depth value is then input into the classification network of the preset model so that the classification network performs binary classification, that is, determines whether the target object contained in the image sequence to be detected is living or non-living. The model structure of the preset model in this embodiment can be as follows: Figure 3 As shown. For the parts not described in detail in this embodiment, please refer to Figure 1 Description of the illustrated embodiment.

[0102] In this embodiment, a preset model can be used to sequentially extract image features, estimate depth values, and identify the target object type. Because information about the image sequence to be detected is extracted simultaneously in both the temporal and spatial dimensions, the extracted image features contain richer information and a wider range of dimensions, thereby ensuring accurate target object type identification.

[0103] based on Figures 2 and 3 In the embodiment shown, the prediction model can optionally be Figure 4 The method shown in FIG. 1 is to extract image features of the image sequence to be detected, and the method may include the following steps:

[0104] S301, performing three-dimensional convolution on the image sequence to be detected through the first feature extraction network in the preset model to obtain the first image feature.

[0105] S302: Perform three-dimensional convolution on the first image feature through a second feature extraction network in a preset model to obtain a second image feature.

[0106] According to the description of the above embodiment, it can be known that the information of the image sequence to be detected in the time dimension and the space dimension can be extracted simultaneously by using a three-dimensional convolution method. Figure 5The prediction model shown in the figure can perform three-dimensional convolution on the input image sequence to be detected through the first feature extraction network in the preset model to obtain first image features. The first image features are then further input into the second feature extraction network in the prediction model, and the second feature extraction network performs three-dimensional convolution on the first image features to obtain second image features. The first feature extraction network and the second feature extraction network are both composed of at least one convolutional layer.

[0107] It should be noted that the first image feature of the image sequence to be detected can be obtained through a smaller number of three-dimensional convolutions; and the second image feature can be obtained through a larger number of three-dimensional convolutions. In this case, the first image feature can be considered as a shallow feature of the image sequence to be detected, which can usually reflect the color information, texture feature information, etc. of the image sequence to be detected; correspondingly, the second image feature can be considered as a deep feature of the image sequence to be detected, which can usually reflect the semantic information of the image sequence to be detected. It can be seen that the multiple image features extracted are actually image features at different levels.

[0108] S303: Using a fusion network in a preset model, the first image feature and the second image feature are fused to obtain image features of the image sequence to be detected.

[0109] Finally, the image features at different levels are input into the fusion network in the preset model to fuse the two. Feature fusion is simply feature concatenation. The fusion result is the image feature of the image sequence to be detected in step 102.

[0110] In this embodiment, different levels of image features can be extracted through different feature extraction networks, and then the image features of different levels are fused. The obtained fusion result includes information on the time dimension and spatial dimension of the image sequence to be detected, as well as information on different aspects such as color, texture, and semantics of the image sequence to be detected, making the content contained in the fusion result richer, thereby further ensuring the accuracy of similar recognition of the target object.

[0111] Optionally, due to the different number of three-dimensional convolution calculations, the size of the first image feature is larger than that of the second image feature. In order to ensure the realization of feature fusion, based on Figure 6 The prediction model shown can also perform the following processing on the first image feature:

[0112] The first image feature is input into a sampling network within a preset model, where it downsamples the first image feature to obtain a sampling result. The sampling network is positioned between the first and second feature extraction networks. Downsampling ensures that the sampling result and the second image feature have the same size. The sampling result and the second image feature are then input into a fusion network within the preset model, where they are fused to obtain the image features of the image sequence to be detected.

[0113] It needs to be explained that: Figures 4 to 6 The preset model provided in the embodiment shown includes two feature extraction networks. However, the present invention does not limit the number of feature extraction networks, which can also be 3 or more. When there are 3 feature extraction networks, the structure of the preset model can be as follows: Figure 7 shown.

[0114] In addition, regardless of the number of feature extraction networks in the preset model, the features extracted by any feature extraction network M1 in the multiple feature extraction networks will be input into two different networks in the preset model. That is, the features extracted by the feature extraction network M1 will be input into the fusion network at the same time as the next-level feature extraction network M2. Optionally, a downsampling network is also provided after each feature extraction network. Downsampling can ensure that the features input into the fusion network are of the same size.

[0115] Among them, the first feature extraction network and the sampling network together constitute a feature extraction module. Of course, the first feature extraction network and the sampling network also constitute a feature extraction module.

[0116] According to the description of the above embodiments, the preset model plays an important role in determining the type of the target object. Figure 7 Flowchart of another data processing method provided by the embodiment of the present invention. This method is actually a model training method for training the above-mentioned preset model. In this embodiment, the structure of the preset model can be as follows Figures 5 and 6 As shown. Figure 8 As shown, the method may include the following steps:

[0117] S401: Input the training image sequence into the first feature extraction network, so that the first feature extraction network performs three-dimensional convolution on the training image sequence to obtain a third image feature.

[0118] S402: Input the third image feature into the second feature extraction network, so that the second feature extraction network performs three-dimensional convolution on the third image feature to obtain a fourth image feature.

[0119] S403: Input the third image feature and the fourth image feature into a fusion network, so that the fusion network performs feature fusion on the third image feature and the fourth image feature.

[0120] S404: Input the feature fusion result into a depth estimation network, and the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence.

[0121] S405 , adjusting model parameters of a preset model according to the input depth image sequence and the predicted depth image sequence corresponding to the training image sequence.

[0122] Training image sequences can be obtained through internet searches or from historical attendance clocking-in images. A corresponding depth image sequence, i.e., the input depth image sequence, can also be obtained for each training image sequence. Furthermore, the training image sequences can be annotated to determine whether the target objects contained in them are living. In this case, the annotation type is actually the real type.

[0123] The training image sequence is then input into the first feature extraction network to extract the third image feature; the third image feature is then input into the second feature extraction network to extract the fourth image feature. The third and fourth image features are features at different levels, reflecting different aspects of the image sequence to be detected. The third and fourth image features are then input into the fusion network to obtain a fusion result. The depth estimation network in the preset model then performs depth estimation on the fusion result, outputting a predicted depth image sequence. At this point, the preset model can directly predict the type of the target object based on the range of depth values ​​contained in the preset depth image sequence. This type is effectively the predicted type.

[0124] At the same time, a loss value can be calculated based on the predicted depth image sequence and the input depth image sequence, and the model parameters of the preset model can be adjusted based on the loss value. Specifically, the parameters of the first feature extraction network, the second feature extraction network, the fusion network, and the depth estimation network are adjusted.

[0125] Optionally, the depth estimation network can also be trained separately. In this case, the parameters of the depth estimation network can be fixed, and the above-calculated loss value is used to adjust the parameters of the first feature extraction network, the second feature extraction network, and the fusion network in the prediction model.

[0126] In addition, in practical applications, the input depth image sequence is usually the same size as the training sample sequence, which is larger than the size of the predicted depth image sequence. Therefore, an optional approach is to upsample the predicted depth image sequence to make it the same size as the input depth image sequence, and then calculate the loss value based on the upsampled input depth image sequence and the predicted depth image sequence.

[0127] However, another optional and more commonly used method is to directly downsample the input depth image sequence so that the downsampled input depth image sequence and the predicted depth image sequence have the same size, and further calculate the loss value based on the downsampled input depth image sequence and the predicted depth image sequence.

[0128] Optionally, the predicted depth image sequence output by the depth estimation network can also be input into the classification network in the preset model. The classification network predicts the type of the target object contained in the training image sequence. This type is actually the predicted type. Finally, a loss value is calculated based on the actual type of the target object annotated by the user and the predicted type output by the classification network. The parameters of the preset model are further adjusted based on this loss value. In this case, the parameters of each network in the model are actually adjusted.

[0129] In this embodiment, based on Figure 5 or Figure 6 In the preset model shown, the first and second feature extraction networks extract image features at different levels. After fusing these two features, the fusion result includes not only temporal and spatial information about the image sequence under test, but also information about color, texture, semantics, and other aspects of the image sequence under test. This rich information in the fusion result ensures the effectiveness of the preset model training, further ensuring the accuracy of subsequent object type recognition.

[0130] like Figure 8 The preset model training method shown can be found in Figures 1 to 7 The data processing method shown above can be executed based on the data processing method, or it can be executed independently of the data processing method. In actual applications, some users, such as enterprise managers, may need to train a preset model. After obtaining the preset model, it can be deployed on the corresponding hardware device to establish an internal company permission application system, such as an attendance clocking system or an access control system.

[0131] In order to meet the user's model training needs, Figure 9 This is a flow chart of a service providing method provided by an embodiment of the present invention. The service providing method provided by an embodiment of the present invention can be executed by a training device. It is understandable that the training device can be implemented as software, or a combination of software and hardware. This training device can be understood as a service platform that provides model training services. Figure 9 As shown, the method may include the following steps:

[0132] S501: Receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence.

[0133] S502: A first feature extraction network in a preset model of a training image sequence is used to simultaneously extract information of the training image sequence in a time dimension and a space dimension by the first feature extraction network to obtain a first image feature.

[0134] S503: Input the first image feature into a second feature extraction network in a preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in a time dimension and a space dimension to obtain a second image feature.

[0135] S404: Input the first image feature and the second image feature into a fusion network in a preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature.

[0136] S505 , inputting the feature fusion result into a depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence.

[0137] S506 : Adjust model parameters of a preset model according to the input depth image sequence and the predicted depth image sequence.

[0138] S507, output model parameters.

[0139] Specifically, a training image sequence and its corresponding input depth image sequence can be collected by the user. After the training image sequence is input into a preset model and sequentially passes through a feature extraction network, a fusion network, and a depth estimation network, the depth estimation network can output a preset depth image sequence and directly predict the target object type based on the range of depth values ​​contained in the preset depth image sequence. Each feature extraction network can employ a three-dimensional convolution approach when extracting the first image feature and the second image feature, respectively.

[0140] At this point, the model parameters of the preset model can be adjusted based on the preset depth image sequence output by the depth estimation network and the input depth image sequence obtained by the user to achieve convergence of the preset model. The model parameters of the preset model are also displayed to the user simultaneously, which means that the training device provides model training services to the user. The training device can also be understood as the processing device in the above embodiments.

[0141] It should be noted that the first image feature in this embodiment is Figure 8 The third image feature in the embodiment shown; the second image feature is Figure 8 Fourth image feature in the illustrated embodiment.

[0142] Optionally, the preset depth image sequence output by the depth estimation network can be further input into the classification network in the preset model so that the classification network can classify the target object. In this case, the model parameters of the preset model can be adjusted according to the type of target object annotated by the user and the type of target object output by the classification network, so that the preset model converges. The structure of the preset model can be as follows: Figure 5 or Figure 6 shown.

[0143] For the parts not described in detail in this embodiment, please refer to Figure 8 The implementation process and technical effects of this technical solution can be found in Figure 8 The description in the illustrated embodiment will not be repeated here.

[0144] In addition to the above-mentioned preset model training service, for the above-mentioned attendance punching and work area usage scenarios, users may also have the need for liveness judgment services, and can also follow the above Figures 1 to 7 The provided embodiment provides a target object type detection service for users, wherein the type of the target object can reflect whether it is a living object.

[0145] In the case of providing type detection services, Figure 10 This is a flow chart of another service providing method provided by an embodiment of the present invention. The service providing method provided by an embodiment of the present invention can be executed by a processing device. The processing device can be understood as a service platform that provides type detection services. Figure 10 As shown, the method may include the following steps:

[0146] S601 : In response to an image acquisition operation triggered by a user, an image sequence to be detected is acquired.

[0147] S602 : extract information of the image sequence to be detected in the time dimension and the space dimension simultaneously to obtain image features of the image sequence to be detected.

[0148] S603: Output the depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected.

[0149] Specifically, a user can trigger an image acquisition operation on the processing device to cause a camera configured on the processing device to capture a sequence of images to be detected. Depending on the application scenario, the image acquisition operation can be a user-triggered camera activation operation or the permission verification operation described in the above embodiment.

[0150] Then, after feature extraction and depth value prediction of the image sequence to be detected, the depth value can be obtained and output. The depth value is specifically expressed as a depth value matrix. Alternatively, the user can directly determine the type of the target object in the image sequence to be detected based on whether the range of variation between the numerical values ​​of the elements in the depth value matrix is ​​greater than a preset range, where the type reflects whether the target object is alive. In this way, the processing device does not directly output the type of the target object, but outputs an indirect result that can reflect the type of the target object, that is, it indirectly provides the user with a target object type detection service.

[0151] Alternatively, the processing device may automatically determine the type of the target object based on the depth value while outputting the depth value to the user. The process of determining the type by the processing device is actually the same as the manual determination process described above.

[0152] Another optional method, according to the above description, since the feature extraction and depth value estimation of the image sequence to be detected can be completed by the preset model, the estimated depth value can be input into the classification network in the preset model, so that the classification network can perform classification according to the estimated depth value to obtain the type of the target object.

[0153] In both of the above methods, the processing device can directly output the target object type, thereby providing the user with a target object type detection service. Furthermore, the user can compare the target object type output by the processing device with the target object type determined based on the depth value output by the processing device, thereby completing a type verification process.

[0154] In addition, for the parts not described in detail in this embodiment, please refer to Figures 1 to 7 The implementation process and technical effects of this technical solution can be found in Figures 1 to 7 The description in the illustrated embodiment will not be repeated here.

[0155] For ease of understanding, the specific implementation of the data processing method provided above is illustrated in conjunction with the attendance punching scenario. Figure 11 understand.

[0156] First, a company employee, A, can trigger a permission request on the processing device. This means they clock in on the clock-in interface provided by the clock-in device. At this point, the camera on the processing device activates and begins capturing their face, generating a sequence of images to be detected. Employee A's face in this sequence serves as the target object. Then, through three-dimensional convolution, information from the image sequence in both the temporal and spatial dimensions—that is, the image features—are extracted.

[0157] The image features of the image sequence to be detected can be extracted by a preset model containing multiple feature extraction networks. The model structure of the preset model can be Figure 5 shown.

[0158] Specifically, the image sequence to be detected is fed into the first feature extraction network of the preset model to extract the first image features. The first image features are then fed into the second feature extraction network of the preset model to obtain the second image features. The first and second image features are then fed into the fusion network, and the fusion results become the image features of the image sequence to be detected.

[0159] Optionally, in order to ensure the accuracy of the fusion between the first image feature and the second image feature, a sampling network is further provided after the first feature extraction network to downsample the first image feature and fuse the downsampled result with the second image feature. In this case, the structure of the preset model can be Figure 6 shown.

[0160] Next, the depth values ​​of the image features in the image sequence to be tested can be estimated. This depth estimation can be performed by the depth estimation network in the preset model. Specifically, the estimated depth values ​​form a depth value matrix. If the range of variation between the values ​​of the elements in the matrix exceeds the preset range, the classification network in the preset model will output that Employee A's face is alive. Finally, the clock-in device will further identify the employee in the image sequence as Employee A based on the image sequence to be tested, allowing Employee A to complete the clock-in process. The clock-in device can then output a message indicating that the clock-in was successful.

[0161] Similarly, the specific implementation of the data processing method provided above can be illustrated in conjunction with the retail scenario. Figure 12 understand.

[0162] A user can purchase items on a processing device, such as a vending machine. A payment interface, such as a virtual button with the words "Please Pay," will then appear. Clicking this button triggers payment authorization verification, and the interface then changes to an image capture interface, enabling the capture of the image sequence to be detected, namely the user's facial image sequence.

[0163] Next, the pre-set model is used to extract features from the image sequence to be detected, and depth values ​​are estimated for the extracted image features. Ultimately, the vending machine can determine based on the depth values ​​that the target object in the image sequence is alive. Based on the user's identification information, the machine can then obtain the payment account corresponding to that identification information and debit the payment from the account, effectively completing the payment. A successful payment notification will be displayed on the vending machine's interface.

[0164] The specific implementation process of this scenario can also be seen Figures 1 to 7 as well as Figure 11 Related description in .

[0165] The data processing devices of one or more embodiments of the present invention are described in detail below. Those skilled in the art will appreciate that these data processing devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0166] Figure 13 A structural diagram of a data processing device provided by an embodiment of the present invention is shown in FIG. Figure 13 As shown, the device includes:

[0167] The acquisition module 11 is used to acquire a sequence of images to be detected.

[0168] The extraction module 12 is configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected.

[0169] The depth value determination module 13 is configured to determine the depth value of the image feature.

[0170] The type determination module 14 is configured to determine the type of the target object contained in the to-be-detected image sequence according to the depth value.

[0171] Optionally, the extraction module 12 is specifically configured to: input the image sequence to be detected into a preset model, so as to extract image features of the image sequence to be detected by the preset model.

[0172] The type of the target object reflects whether the target object is alive.

[0173] Optionally, the extraction module 12 is specifically configured to: perform three-dimensional convolution on the image sequence to be detected through a first feature extraction network in the preset model to obtain a first image feature;

[0174] Performing three-dimensional convolution on the first image feature through a second feature extraction network in the preset model to obtain a second image feature;

[0175] The first image feature and the second image feature are fused through the fusion network in the preset model to obtain the image feature of the image sequence to be detected.

[0176] Optionally, the extraction module 12 is specifically used to: downsample the first image features through the sampling network in the preset model; and obtain the image features of the image sequence to be detected from the sampling results and the second image features through the fusion network in the preset model.

[0177] Optionally, the depth value determination module 13 is specifically configured to: input the image features of the to-be-detected image sequence into a depth estimation network in the preset model, so that the depth estimation network estimates the depth value of the image feature.

[0178] The type determination module 14 is configured to input the depth value into a classification network in the preset model, so that the classification network determines the type of the target object.

[0179] Optionally, the acquisition module 11 is specifically configured to: acquire the image sequence to be detected in response to a permission verification operation triggered by a user.

[0180] The device further includes: an identification module 21 and a validity determination module 22 .

[0181] The recognition model 21 is used to identify the identification information of the target object if the target object is a living type.

[0182] The validity determination module 22 is configured to determine the validity of the permission verification operation according to the identification information.

[0183] Optionally, the permission verification operation includes an attendance clocking-in operation.

[0184] The device further comprises: a display module 23 for displaying a prompt message indicating that the attendance punching operation is valid.

[0185] Optionally, the authority verification operation includes an access control authority verification operation.

[0186] The display module 23 further includes: if the access control permission verification operation is valid, displaying prompt information that the user has the access control permission. Optionally, the permission verification operation includes a payment permission verification operation.

[0187] The display module 23 further includes: if the verification operation of the payment authority is valid, displaying prompt information of successful payment.

[0188] Optionally, the device further includes: an input module 24 and an adjustment module 25 .

[0189] The input module 24 is configured to input a training image sequence into the first feature extraction network, so that the first feature extraction network performs a three-dimensional convolution on the training image sequence to obtain a third image feature;

[0190] Inputting the third image feature into the second feature extraction network, so that the second feature extraction network performs three-dimensional convolution on the third image feature to obtain a fourth image feature;

[0191] Inputting the third image feature and the fourth image feature into the fusion network, so that the fusion network performs feature fusion on the third image feature and the fourth image feature;

[0192] Inputting the feature fusion result into the depth estimation network, and the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence;

[0193] The adjustment module 25 is configured to adjust the model parameters of the preset model according to the input depth image sequence corresponding to the training image sequence and the predicted depth image sequence.

[0194] Optionally, the input module 24 is further configured to: input the predicted depth image sequence into the classification network, so that the classification network outputs the type of the target object contained in the training image sequence.

[0195] The adjustment module 25 is further configured to adjust the model parameters of the preset model according to the type of the target object labeled by the user and the type of the target object output by the classification network.

[0196] Figure 13 The device shown can perform Figures 1 to 8 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figures 1 to 8 The implementation process and technical effects of this technical solution can be found in Figures 1 to 8 The description in the illustrated embodiment will not be repeated here.

[0197] The above describes the internal functions and structure of the data processing device. In a possible design, the structure of the data processing device can be implemented as an electronic device, such as Figure 14 As shown, the electronic device may include: a processor 31 and a memory 32. The memory 32 is used to store the data that supports the electronic device to execute the above Figures 1 to 8 The program of the data processing method provided in the illustrated embodiment, the processor 31 is configured to execute the program stored in the memory 32 .

[0198] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 31, can implement the following steps:

[0199] Obtaining a sequence of images to be detected;

[0200] Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0201] determining a depth value of the image feature;

[0202] The type of the target object contained in the to-be-detected image sequence is determined according to the depth value.

[0203] Optionally, the processor 31 is further configured to execute the aforementioned Figures 1 to 8 All or part of the steps in the illustrated embodiments.

[0204] The structure of the electronic device may further include a communication interface 33 for the electronic device to communicate with other devices or a communication network.

[0205] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figures 1 to 8 The procedures involved in the data processing method in the method embodiment shown.

[0206] The service providing apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these service providing apparatuses can be configured using commercially available hardware components through the steps taught in this solution.

[0207] Figure 15 A schematic diagram of the structure of a service providing device provided by an embodiment of the present invention is shown in FIG. Figure 15 As shown, the device includes:

[0208] The acquisition module 41 is configured to acquire a sequence of images to be detected in response to an image acquisition operation triggered by a user.

[0209] The extraction module 42 is configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected.

[0210] The output module 43 is configured to output a depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected.

[0211] Optionally, the apparatus further includes: a determination module 44, configured to determine the type of the target object contained in the to-be-detected image sequence according to the depth value.

[0212] The output module 43 is configured to output the type of the target object.

[0213] The type of the target object reflects whether the target object is alive.

[0214] Figure 15 The device shown can perform Figure 10 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figure 10 The implementation process and technical effects of this technical solution can be found in Figure 10 The description in the illustrated embodiment will not be repeated here.

[0215] The above describes the internal functions and structure of the service providing device. In a possible design, the structure of the service providing device can be implemented as an electronic device, such as Figure 16 As shown, the electronic device may include: a processor 51 and a memory 52. ​​The memory 52 is used to store the data that supports the electronic device to execute the above Figure 10 The program of the service providing method provided in the illustrated embodiment, the processor 51 is configured to execute the program stored in the memory 52 .

[0216] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 51, can implement the following steps:

[0217] In response to an image acquisition operation triggered by a user, acquiring a sequence of images to be detected;

[0218] Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected;

[0219] Output a depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected.

[0220] Optionally, the processor 51 is further configured to execute the aforementioned Figure 9 All or part of the steps in the illustrated embodiments.

[0221] The structure of the electronic device may further include a communication interface 53 for the electronic device to communicate with other devices or a communication network.

[0222] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figure 10 The procedures involved in the service provision method in the method embodiment shown.

[0223] Figure 17 A structural diagram of another service providing device provided by an embodiment of the present invention, such as Figure 17 As shown, the device includes:

[0224] The receiving module 61 is configured to receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence.

[0225] An input module 62 is configured to input the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in both the temporal and spatial dimensions to obtain first image features;

[0226] Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature;

[0227] Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature;

[0228] The feature fusion result is input into the depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence.

[0229] The adjustment module 63 is configured to adjust the model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence.

[0230] The output module 64 is used to output the model parameters.

[0231] Optionally, the input module 62 is further configured to input the predicted depth image into the classification network, so that the classification network outputs the type of the target object contained in the training image sequence.

[0232] The adjustment module 63 is further configured to adjust the model parameters of the preset model according to the type of the target object labeled by the user and the type of the target object output by the classification network.

[0233] Figure 17 The device shown can perform Figure 9 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figure 9 The implementation process and technical effects of this technical solution can be found in Figure 9 The description in the illustrated embodiment will not be repeated here.

[0234] The above describes the internal functions and structure of the service providing device. In a possible design, the structure of the service providing device can be implemented as an electronic device, such as Figure 18 As shown, the electronic device may include: a processor 71 and a memory 72. The memory 72 is used to store the data that supports the electronic device to execute the above Figure 9 The program of the service providing method provided in the illustrated embodiment, the processor 71 is configured to execute the program stored in the memory 72 .

[0235] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 71, can implement the following steps:

[0236] Receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence;

[0237] Inputting the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in a time dimension and a space dimension to obtain a first image feature;

[0238] Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature;

[0239] Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature;

[0240] Inputting the feature fusion result into the depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence;

[0241] Adjusting model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence;

[0242] The model parameters are output.

[0243] Optionally, the processor 51 is further configured to execute the aforementioned Figure 9 All or part of the steps in the illustrated embodiments.

[0244] The structure of the electronic device may further include a communication interface 73 for the electronic device to communicate with other devices or a communication network.

[0245] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figure 9 The procedures involved in the service provision method in the method embodiment shown.

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data processing method, characterized in that: include: Obtaining a sequence of images to be detected; Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected; determining a depth value of the image feature; Determine the type of the target object contained in the image sequence to be detected according to the depth value, The step of simultaneously extracting information of the image sequence to be detected in both the time dimension and the space dimension includes: The image sequence to be detected is input into a preset model, and a first feature extraction network in the preset model is used to perform three-dimensional convolution on the image sequence to obtain a first image feature; a second feature extraction network in the preset model is used to perform three-dimensional convolution on the first image feature to obtain a second image feature; and a fusion network in the preset model is used to fuse the first image feature and the second image feature to obtain the image feature of the image sequence to be detected.

2. The method according to claim 1, characterized in that The type of the target object reflects whether the target object is a living body.

3. The method according to claim 1, characterized in that The step of fusing the first image feature and the second image feature through the fusion network in the preset model to obtain the image feature of the image sequence to be detected includes: Downsampling the first image feature through a sampling network in the preset model; The image features of the image sequence to be detected are obtained by combining the sampling results and the second image features through the fusion network in the preset model.

4. The method according to claim 1, wherein Determining the depth value of the image feature includes: Inputting the image features of the image sequence to be detected into the depth estimation network in the preset model, so that the depth estimation network estimates the depth value of the image features; The determining, according to the depth value, the type of the target object contained in the image sequence to be detected includes: The depth value is input into a classification network in the preset model so that the classification network determines the type of the target object.

5. The method according to any one of claims 1 to 4, characterized in that The step of obtaining a sequence of images to be detected includes: In response to a permission verification operation triggered by a user, obtaining a sequence of images to be detected; The method further comprises: If the target object is a living body, identifying the identification information of the target object; The validity of the permission verification operation is determined according to the identification information.

6. The method according to claim 5, characterized in that The permission verification operation includes an attendance clocking operation; the method further includes: If the attendance clock-in operation is valid, a prompt message indicating successful clock-in is displayed.

7. The method according to claim 5, characterized in that The permission verification operation includes a verification operation of access control permission; the method further includes: If the verification operation of the access control authority is valid, a prompt message indicating that the user has the access control authority is displayed.

8. The method according to claim 5, characterized in that The permission verification operation includes a payment permission verification operation; the method further includes: If the payment permission verification operation is valid, a prompt message indicating successful payment will be displayed.

9. The method according to claim 4, characterized in that The method further comprises: Inputting a training image sequence into the first feature extraction network, so that the first feature extraction network performs three-dimensional convolution on the training image sequence to obtain a third image feature; Inputting the third image feature into the second feature extraction network, so that the second feature extraction network performs three-dimensional convolution on the third image feature to obtain a fourth image feature; Inputting the third image feature and the fourth image feature into the fusion network, so that the fusion network performs feature fusion on the third image feature and the fourth image feature; Inputting the feature fusion result into the depth estimation network, and the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence; Adjusting model parameters of the preset model according to an input depth image sequence corresponding to the training image sequence and the predicted depth image sequence.

10. The method according to claim 9, characterized in that The method further comprises: Inputting the predicted depth image sequence into the classification network, so that the classification network outputs the type of the target object contained in the training image sequence; The model parameters of the preset model are adjusted according to the type of the target object labeled by the user and the type of the target object output by the classification network.

11. A service providing method, characterized in that: Applications include: In response to an image acquisition operation triggered by a user, acquiring a sequence of images to be detected; Simultaneously extracting information of the image sequence to be detected in the time dimension and the space dimension to obtain image features of the image sequence to be detected; Outputting a depth value of the image feature, where the depth value reflects the type of the target object contained in the image sequence to be detected; The step of simultaneously extracting information of the image sequence to be detected in both the time dimension and the space dimension includes: The image sequence to be detected is input into a preset model, and a first feature extraction network in the preset model is used to perform three-dimensional convolution on the image sequence to obtain a first image feature; a second feature extraction network in the preset model is used to perform three-dimensional convolution on the first image feature to obtain a second image feature; and a fusion network in the preset model is used to fuse the first image feature and the second image feature to obtain the image feature of the image sequence to be detected.

12. The method according to claim 11, characterized in that The method further comprises: determining the type of the target object contained in the image sequence to be detected according to the depth value; Outputs the type of the target object.

13. The method according to claim 12, characterized in that The type of the target object reflects whether the target object is a living body.

14. A service providing method, characterized in that: include: Receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence; Inputting the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in a time dimension and a space dimension to obtain a first image feature; Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature; Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature; Inputting the feature fusion result into a depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence; Adjusting model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence; The model parameters are output.

15. The method according to claim 14, characterized in that The method further comprises: Inputting the predicted depth image into a classification network in the preset model, so that the classification network outputs the type of the target object contained in the training image sequence; The model parameters of the preset model are adjusted according to the type of the target object labeled by the user and the type of the target object output by the classification network.

16. A data processing device, characterized in that: include: An acquisition module, used for acquiring an image sequence to be detected; An extraction module, configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected; A depth value determination module, configured to determine a depth value of the image feature; a type determination module, configured to determine the type of the target object contained in the image sequence to be detected according to the depth value; The step of simultaneously extracting information of the image sequence to be detected in both the time dimension and the space dimension includes: The image sequence to be detected is input into a preset model, and a first feature extraction network in the preset model is used to perform three-dimensional convolution on the image sequence to obtain a first image feature; a second feature extraction network in the preset model is used to perform three-dimensional convolution on the first image feature to obtain a second image feature; and a fusion network in the preset model is used to fuse the first image feature and the second image feature to obtain the image feature of the image sequence to be detected.

17. An electronic device, characterized in that: include: A shooting component, a memory, and a processor; wherein the shooting component is used to shoot a sequence of images to be detected; The memory stores executable codes, and when the processor executes the executable codes, the processor executes the data processing method according to any one of claims 1 to 10.

18. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the data processing method according to any one of claims 1 to 10.

19. A service providing device, characterized in that: include: An acquisition module, configured to acquire a sequence of images to be detected in response to an image acquisition operation triggered by a user; An extraction module, configured to simultaneously extract information of the image sequence to be detected in both the time dimension and the space dimension to obtain image features of the image sequence to be detected; an output module, configured to output a depth value of the image feature, wherein the depth value reflects the type of the target object contained in the image sequence to be detected; The step of simultaneously extracting information of the image sequence to be detected in both the time dimension and the space dimension includes: The image sequence to be detected is input into a preset model, and a first feature extraction network in the preset model is used to perform three-dimensional convolution on the image sequence to obtain a first image feature; a second feature extraction network in the preset model is used to perform three-dimensional convolution on the first image feature to obtain a second image feature; and a fusion network in the preset model is used to fuse the first image feature and the second image feature to obtain the image feature of the image sequence to be detected.

20. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the service providing method according to any one of claims 11 to 13.

21. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the service providing method according to any one of claims 11 to 13.

22. A service providing device, characterized in that: include: A receiving module, configured to receive a training image sequence input by a user and an input depth image sequence corresponding to the training image sequence; an input module, configured to input the training image sequence into a first feature extraction network in a preset model, so that the first feature extraction network simultaneously extracts information of the training image sequence in a temporal dimension and a spatial dimension to obtain a first image feature; Inputting the first image feature into a second feature extraction network in the preset model, so that the second feature extraction network simultaneously extracts information of the first image feature in the time dimension and the spatial dimension to obtain a second image feature; Inputting the first image feature and the second image feature into a fusion network in the preset model, so that the fusion network performs feature fusion on the first image feature and the second image feature; Inputting the feature fusion result into a depth estimation network, so that the depth estimation network outputs a predicted depth image sequence corresponding to the training image sequence; An adjustment module, configured to adjust model parameters of the preset model according to the input depth image sequence and the predicted depth image sequence; An output module is used to output the model parameters.

23. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores an executable code, and when the executable code is executed by the processor, the processor executes the service providing method according to claim 14 or 15.

24. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the service providing method according to claim 14 or 15.

Images