Image data processing method and apparatus, device, and storage medium

Abstract

An image data processing method, applied to a recognition device. The recognition device comprises an image signal processor. The method comprises: acquiring an original part image comprising a recognized part, and recognition performance parameters respectively corresponding to N recognition network models, N being a positive integer (S101); on the basis of the recognition performance parameters respectively corresponding to the N recognition network models, determining from among the N recognition network models a first recognition network model, the recognition performance parameter of which does not satisfy a processor resource occupation condition (S102); if image resolution to which the first recognition network model is adapted is different from the image resolution of the original part image, on the basis of the image resolution to which the first recognition network model is adapted, adjusting, by means of the image signal processor, the image resolution of the original part image, so as to obtain an adjusted part image corresponding to the first recognition network model (S103); and determining a preprocessed image of the adjusted part image on the basis of the adjusted part image, and transmitting the preprocessed image of the adjusted part image to a central processing unit, the central processing unit being used for invoking the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain a first object identifier corresponding to the adjusted part image, and the first object identifier being used for reflecting an object to which the recognized part belongs (S104).

Description

Image data processing methods, apparatus, devices and storage media

[0001] Related applications

[0002] This application claims priority to Chinese patent application filed on October 23, 2024, with application number 202411490963.4, entitled "Image Data Processing Method, Apparatus, Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to fields such as Internet technology and artificial intelligence, and in particular to an image data processing method, apparatus, device and storage medium. Background Technology

[0004] With the development of computer technology, the application of body part recognition technology is becoming more and more widespread. It can be applied to various scenarios, such as payment scenarios or access control management scenarios. Through body part recognition, user identity can be verified.

[0005] Currently, in the process of body part recognition, computer equipment acquires images of the body part to be recognized, and then uses a recognition network model to identify the body part, thereby completing the identity verification. However, the body part images usually need to be preprocessed by the Central Processing Unit (CPU) before they can be used by the recognition network model, which increases the processing time of the CPU. Summary of the Invention

[0006] This application provides an image data processing method, apparatus, device, and storage medium.

[0007] This application provides an image data processing method applied to a recognition device, wherein the recognition device includes an image signal processor, and the method includes:

[0008] Obtain the original image of the region to be identified, and the recognition performance parameters corresponding to N recognition network models; N is a positive integer.

[0009] Based on the recognition performance parameters corresponding to the above N recognition network models, determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models;

[0010] If the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, then the image signal processor adjusts the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model; and

[0011] Based on the image of the adjusted part, a preprocessed image of the adjusted part is determined and transmitted to the central processing unit. The central processing unit calls the first recognition network model to perform part recognition on the preprocessed image of the adjusted part to obtain a first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

[0012] One embodiment of this application provides an image data processing apparatus for use in a recognition device, wherein the recognition device includes an image signal processor, and the apparatus includes:

[0013] The acquisition module is used to acquire the original image of the region to be identified, as well as the recognition performance parameters corresponding to N recognition network models; N is a positive integer.

[0014] The determination module is used to determine, based on the recognition performance parameters corresponding to the above N recognition network models, the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models;

[0015] The adjustment module is used to adjust the image resolution of the original part image according to the image resolution adapted by the first recognition network model if the image resolution adapted by the first recognition network model is different from the image resolution of the original part image, so as to obtain the adjusted part image corresponding to the first recognition network model.

[0016] The transmission module is used to determine a preprocessed image of the adjusted part image based on the adjusted part image, and transmit the preprocessed image of the adjusted part image to the central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain a first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

[0017] One embodiment of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0018] One embodiment of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0019] One embodiment of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0020] Details of one or more embodiments of this application are set forth in the following drawings and description. Other features, objects, and advantages of this application will become apparent from the specification, drawings, and claims. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the published drawings without creative effort.

[0022] Figure 1 is a schematic diagram of an image data processing system provided in this application;

[0023] Figure 2a is a schematic diagram of the structure of an identification device provided in this application;

[0024] Figure 2b is a schematic diagram of the processing flow of the preprocessing module of an identification device provided in this application;

[0025] Figure 3 is a flowchart illustrating an image data processing method provided in this application;

[0026] Figure 4 is a flowchart illustrating another image data processing method provided in this application;

[0027] Figure 5 is an exposure timing diagram of a color camera provided in this application;

[0028] Figure 6 is an exposure timing diagram of an infrared camera provided in this application;

[0029] Figure 7 is a flowchart illustrating another image data processing method provided in this application;

[0030] Figure 8 is a schematic diagram of another image data processing method provided in this application;

[0031] Figure 9 is a schematic diagram of the structure of an image data processing device provided in an embodiment of this application;

[0032] Figure 10 is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] To facilitate a clearer understanding of this application, the image data processing system implementing this application is first introduced, as shown in Figure 1. This image data processing system includes a server and a terminal cluster. The terminal cluster can include one or more terminals; the number of terminals is not limited here. As shown in Figure 1, taking a terminal cluster containing four terminals as an example, the terminal cluster specifically includes a first terminal, a second terminal, a third terminal, and a fourth terminal. It is understood that the first terminal, the second terminal, the third terminal, and the fourth terminal can all connect to the server via a network, so that each terminal can interact with the server for data exchange through the network connection.

[0035] Understandably, a server can be a single physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud knowledge base, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0036] Specifically, the terminal can refer to vending machines, vehicle-mounted terminals, smartphones, tablets, laptops, desktop computers, smart speakers, speakers with screens, smart TVs, smartwatches, etc., but is not limited to these. The terminals and servers can be connected directly or indirectly via wired or wireless communication. Furthermore, the number of terminals and servers can be one or at least two; this application does not impose any limitation on this.

[0037] Each terminal can be used to capture images of the identified body part, obtaining raw images of that part. Each terminal can include a recognition application, which can call a recognition network model to perform part recognition on the raw image or a pre-processed image corresponding to the raw image, obtaining an object identifier. This object identifier can be used to indicate the object to which the identified part belongs. The recognition application can be an access control application, a payment application, a transportation application, a shopping application, etc., and can be a standalone application, a web application, or a mini-program within a host application.

[0038] The identification area can be the palm, face, fingers, etc.

[0039] In this context, "server" can refer to a device that provides backend services for the recognition application. For example, the server could be used to maintain the application. It can also be used to train N recognition network models, which can be models used to recognize images of body parts (such as pre-processed images), where N is a positive integer. Specifically, the recognition network model can refer to convolutional neural network models, recurrent neural network models, support vector machines, pre-trained models, etc.

[0040] In one embodiment, after the server trains the recognition network model, it can store the model in the server's local memory. Each terminal can use its recognition application to access the recognition network model in the server's local memory to perform part recognition on the part image and obtain the object identifier corresponding to the recognized part. The server can then authenticate the object corresponding to the recognized part based on the object identifier, obtain the verification result, and return the verification result to the terminal so that the terminal can display the verification result on its recognition application.

[0041] In one embodiment, after the server trains the recognition network model, it can send the model to various terminals, which can then store it in their local memory. The recognition application calls the network model from its local memory to perform part recognition on the image, obtaining the object identifier corresponding to the identified part. This object identifier is then sent to the server, which can use it to authenticate the object, obtain the verification result, and return it to the terminal so that the terminal can display the result on its recognition application.

[0042] In one embodiment, after the server trains the recognition network model, it can send the model to a cloud server, which can then store it in its local storage. Each terminal can use its recognition application to access the recognition network model in the cloud server's local storage to perform part recognition on the part image, obtain the object identifier corresponding to the recognized part, and send the object identifier to the server. The server can then authenticate the object corresponding to the recognized part based on the object identifier, obtain the verification result, and return the verification result to the terminal so that the terminal can display the verification result on its recognition application.

[0043] In one embodiment, the image data processing method provided in this application can be executed by any terminal in the terminal cluster in Figure 1, or by the server in Figure 1, or by the terminals and the server in the terminal cluster in Figure 1 working together. The device used to execute the image data processing method in this application can be collectively referred to as a recognition device.

[0044] Please refer to Figure 2a, which is a schematic diagram of the structure of an identification device provided in an embodiment of this application. The identification device may include an image signal processing module 21a, a preprocessing module 22a, and an alignment module 23a. The image signal processing module 21a can support part images of various resolutions, so that a suitable data stream (i.e., preprocessed image) can be selected according to the needs, avoiding unnecessary high-resolution images.

[0045] The image signal processing module 21a may include a data processing submodule 211b, a switching submodule 212b, and an output submodule 213b. The data processing submodule 211b performs format conversion on the original part image to adapt the format of the converted original part image to the subsequent recognition network model. The switching submodule 212b can dynamically adjust the part image to different resolutions as needed, such as adjusting the part image to match the first recognition network model, where the recognition performance parameters of the recognition device do not meet the processor resource usage requirements. The output submodule 213b can be a multiplexer (MUX) for outputting part images of various resolutions.

[0046] The data processing submodule 211b can also be used to configure corresponding processing pipelines for part images (original part images and adjusted part images) of different resolutions, so as to preprocess the part images through the corresponding processing pipelines to ensure efficient processing at different resolutions.

[0047] The image signal processing module 21a can be implemented by an image signal processor (ISP).

[0048] The preprocessing module 22a can be a module in the image signal processing module 21a, used to screen the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions, so that the image signal processing module 21a can independently design a data stream (i.e., the adjusted part image) for the first recognition network model. That is, the preprocessed image of the adjusted part image can be used as the input of the first recognition network model, which can reduce the amount of computation for data scaling processing (i.e. resolution processing for the original part image) and reduce the burden on CPU processing.

[0049] Specifically, as shown in Figure 2b, the preprocessing module 22a can be used to perform the following steps: S21, obtain the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions. Taking the palm verification scenario as an example, the preprocessing module 22a can use the perf(performance) command to obtain the recognition performance parameters of each recognition network model. Specifically, the preprocessing module 22a can execute the following commands: perf record -g wepalm_app; perf repor, to obtain the recognition performance parameters of each recognition network model. wepalm_app refers to the palm verification application (i.e., the aforementioned recognition application). The perf command is a performance analysis toolset provided by the Linux system, containing various sub-tools that can identify the recognition performance parameters of network models. Further, based on the recognition performance parameters corresponding to each recognition network model, the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions is determined. S22. Control the image signal processing module 21a to output the image of the adjusted part; send the model identifier corresponding to the first recognition network model to the switching submodule 212b in the image signal processing module 21a, so that the switching submodule 212b can adjust the image of the adjusted part to match the first recognition network model. S23. Preprocess the recognition network model in the hardware image accelerator, and preprocess the first recognition network model in the hardware image accelerator. Through the hardware image accelerator, preprocess the image of the adjusted part to obtain the preprocessed image corresponding to the image of the adjusted part.

[0050] The alignment module 23a can read the image to be processed (i.e., the preprocessed image) from the image signal processing module 21a, perform frame synchronization processing on the preprocessed image, and identify the parts of the preprocessed images with synchronization relationships to obtain the final object identifier. This alignment module 23a can be a module of the central processing unit.

[0051] Further, please refer to Figure 3, which is a flowchart illustrating an image data processing method provided in an embodiment of this application.

[0052] As shown in Figure 3, the method may include the following steps:

[0053] S101. Obtain the original part image including the recognition part, and the recognition performance parameters corresponding to N recognition network models respectively; N is a positive integer.

[0054] In this application, the recognition device can capture an image of the recognition area to obtain an original image of the area; or, it can acquire an original image of the area including the recognition area from another device. Recognition performance parameters are obtained for each of the N recognition network models. These performance parameters can refer to the performance of the recognition network model in area recognition; they can include at least one of the processing resources occupied by the recognition network model during area recognition and the processing latency of the recognition network model during area recognition. The processing resources can include at least one of CPU resources, interface resources, and memory resources.

[0055] It should be noted that the original part image here can refer to the part image obtained by photographing the object to be identified, that is, the original part image without image resolution adjustment. The original part image can refer to the part image to be processed. The original part image can include at least one of infrared part images and color part images obtained by photographing the object to be identified. The infrared part image can refer to the part to be identified that is photographed by an infrared camera based on infrared light, and the color part image can refer to the part to be identified that is photographed by a color camera based on natural light.

[0056] It should be noted that the recognition performance parameters corresponding to the N recognition network models can be obtained from the operational log data of the N recognition network models. Specifically, when the N recognition network models are deployed in a recognition device, the recognition device can obtain the operational log data of the N recognition network models and extract the recognition performance parameters corresponding to each recognition network model from the operational log data. When the N recognition network models are deployed in other devices, the recognition device can obtain the recognition performance parameters corresponding to the N recognition network models from those other devices; that is, the recognition performance parameters corresponding to the N recognition network models are obtained by the other devices from the operational log data of the N recognition network models.

[0057] When the identification area is the palm, the identification device can be called a palm identification device or a palm verification device. The palm verification device can refer to a low-cost palm verification device, that is, a palm verification device with a relatively low hardware development (Bill of Materials, BOM) cost.

[0058] It should be noted that when N is an integer greater than 1, the various recognition network models can be different types of recognition network models. Taking the first and second recognition network models as an example, the first recognition network model can be a convolutional neural network model, and the second recognition network model can be a recurrent neural network model. Alternatively, the various recognition network models can refer to the same type of recognition network model, but the image resolutions they are adapted to are different. For example, both the first and second recognition network models are convolutional neural network models, but the first recognition network model is adapted to an image resolution of 128*128, and the second recognition network model is adapted to an image resolution of 256*256.

[0059] S102. Based on the recognition performance parameters corresponding to the above N recognition network models, determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models.

[0060] The processor resource usage conditions include one or more of the following: 1. The processing resources used by the recognition network model in the part recognition process are less than the resource threshold; 2. The recognition network model is not among the top K recognition network models with the most processing resources among the N recognition network models; 3. The processing latency of the recognition network model in the part recognition process is less than the latency threshold; 4. The recognition network model is not among the top M recognition network models with the longest corresponding processing latency among the N recognition network models, where K and M can both be positive integers less than N.

[0061] The resource threshold and latency threshold can be preset, or they can be determined based on the application scenario, such as payment scenarios, access control scenarios, and work attendance scenarios; or they can be dynamically determined based on the recognition performance parameters of each recognition network model.

[0062] It should be noted that the recognition performance parameters of the identification network model not meeting the processor resource usage conditions can refer to one or more of the following: 1. The processing resources used by the identification network model in the part recognition process are greater than or equal to the resource threshold; 2. The identification network model belongs to the top K identification network models with the most processing resources among N identification network models; 3. The processing latency of the identification network model in the part recognition process is greater than or equal to the latency threshold; 4. The identification network model belongs to the top M identification network models with the longest corresponding processing latency among N identification network models.

[0063] The number of first recognition network models can be one or more, and the number of first recognition network models can be a positive integer less than or equal to N.

[0064] In one embodiment, when N is greater than 1, step S102 may include: if a certain recognition network model occupies too much processing resources during the part recognition process, the recognition network model is prone to insufficient processing resources during the part recognition process, causing problems such as stuttering in the part recognition process. Therefore, the recognition device can determine the first recognition network model whose recognition performance parameters do not meet the processor resource occupancy condition based on the processing resources occupied by the recognition network model during the part recognition process. Specifically, the recognition device can obtain the processing resources occupied by the N recognition network models during the part recognition process from the recognition performance parameters corresponding to the N recognition network models respectively, and select K recognition network models from the N recognition network models according to the processing resources corresponding to the N recognition network models respectively; the processing resources corresponding to the K recognition network models are all greater than the processing resources corresponding to the recognition network models that were not selected among the N recognition network models, and K is a positive integer less than N. The K recognition network models are determined as the first recognition network models whose recognition performance parameters do not meet the processor resource occupancy condition. Identifying the first recognition network model that consumes excessive processing resources is beneficial for subsequently adjusting the first recognition network model to obtain the adjusted part image, which helps improve the part recognition efficiency.

[0065] Here, K recognition network models can refer to recognition network models whose processing resources are greater than or equal to the resource threshold among N recognition network models, or K recognition network models can refer to the top K recognition network models among N recognition network models that occupy the most processing resources.

[0066] In particular, when there are multiple types of processing resources, the computer device can sum up the multiple processing resources of each recognition network model, select the top K recognition network models with the most summed processing resources from N recognition network models, and determine the selected K recognition network models as the first recognition network model whose recognition performance parameters do not meet the processor resource occupancy conditions.

[0067] In one embodiment, when N is greater than 1, step S102 may include: if the processing latency of a certain recognition network model in the part recognition process is too large, it indicates that the recognition network model will occupy processing resources for a long time. Therefore, the recognition device can determine the first recognition network model whose recognition performance parameters do not meet the processor resource occupation conditions based on the processing latency of the recognition network model in the part recognition process. Specifically, the recognition device can obtain the processing latency of the above N recognition network models in the part recognition process from the recognition performance parameters corresponding to the above N recognition network models respectively. According to the processing latency corresponding to the above N recognition network models respectively, M recognition network models are selected from the above N recognition network models, and the processing latency corresponding to the above M recognition network models is greater than the processing latency corresponding to the recognition network models that were not selected among the above N recognition network models, where M is a positive integer less than N. The above M recognition network models are determined as the first recognition network models whose recognition performance parameters do not meet the processor resource occupation conditions. By identifying the first recognition network model with excessive processing latency, it is beneficial to subsequently adjust the first recognition network model to obtain the adjusted part image, which is beneficial to reduce the processing resources occupied by the first recognition network model.

[0068] Among them, the M recognition network models can be the top M recognition network models with the largest processing latency among the N recognition network models, or the M recognition network models can be the recognition network models among the N recognition network models whose processing latency is greater than or equal to the latency threshold.

[0069] S103. If the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, the image signal processor adjusts the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0070] The adjusted part image refers to the image obtained by adjusting the resolution of the original part image according to the resolution adapted to the first recognition network model when the image resolution adapted to the first recognition network model is different from the image resolution of the original part image. This image can be used as the input of the first recognition network model.

[0071] In steps S102-S103, the image resolution of the original part image is usually different from the image resolution adapted to the recognition network model. That is, the image resolution of the original part image needs to be adjusted beforehand to obtain the adjusted part image adapted to the recognition network model. If the central processing unit (CPU) adjusts the corresponding adjusted part image for each recognition network model, it increases the CPU's processing time and reduces the efficiency of part recognition. Therefore, an image signal processor can be used to adjust the corresponding adjusted part images for some or all recognition network models.

[0072] Specifically, the recognition device can determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the N recognition network models mentioned above, based on the recognition performance parameters corresponding to each of the N recognition network models. This indicates that the first recognition network model consumes more processing resources and has a larger processing latency, and the central processing unit resources allocated to each recognition network model are limited. If the central processing unit is still used to adjust the corresponding adjustment part image for the first recognition network model, it is easy to cause the first recognition network model to experience stuttering problems during the part recognition process, thus reducing the part recognition performance of the first recognition network model.

[0073] In one possible implementation, for each recognition network model's recognition performance parameter, the CPU resource utilization R can be... CPU Interface resource utilization rate R interface and memory resource utilization R memory By performing a weighted summation, we obtain the comprehensive resource utilization index S, which is expressed by the formula S = αR. CPU +βR interface +γR memory α, β, and γ are weighting coefficients, and α + β + γ = 1. These coefficients can be pre-set according to the system's emphasis on different resources, for example, α = 0.5, β = 0.3, and γ = 0.2. For processing latency, the average processing latency T of the recognition network model during multiple part recognition processes is directly used. avg As a metric, a threshold is set corresponding to the processor resource usage condition, and the processing resource threshold S is defined. threshold and processing delay threshold T threshold If the overall processing resource consumption index S of a certain recognition network model is greater than S... threshold Or average processing delay T avg Greater than T threshold If so, then the recognition network model is determined as the first recognition network model.

[0074] In another possible implementation, for each recognition network model, its overall performance index P can be calculated, using the formula: Where R represents the processing resources used by the recognition network model in the part recognition process. max Let T be the maximum processing resource consumption among the N recognition network models; T is the processing latency of this recognition network model in the part recognition process. max ω1 represents the maximum processing latency among the N recognition network models; ω1 and ω2 are weight coefficients, and ω1 + ω2 = 1. These can be preset according to the system's emphasis on processing resources and processing latency, for example, ω1 = 0.6, ω2 = 0.4. Set the performance indicator threshold P. threshold If the overall performance index P of a certain recognition network model is greater than P threshold If so, then the recognition network model is determined as the first recognition network model.

[0075] Based on this, the recognition device can use an image signal processor to adjust the first recognition network model to obtain the corresponding adjusted part image. Specifically, if the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, the image signal processor can adjust the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model. This helps to reduce the processing time of the central processing unit and improve the efficiency of part recognition.

[0076] In one embodiment, step S103 may include: determining the image size adapted to the first recognition network model by an image signal processor based on the image resolution adapted to the first recognition network model; when the image size adapted to the first recognition network model is larger than the image size of the original part image, the recognition device may adjust the image size of the original part image according to the image size adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0077] For example, when the image size adapted to the first recognition network model is larger than the image size of the original part image, the recognition device can enlarge the image size of the original part image according to the image size adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model. When the image size adapted to the first recognition network model is smaller than the image size of the original part image, the recognition device can reduce the image size of the original part image according to the image size adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0078] In one embodiment, step S103 may include: using an image signal processor to sample the original part image according to the image resolution adapted to the first recognition network model, thereby obtaining the adjusted part image corresponding to the first recognition network model.

[0079] For example, when the image resolution adapted to the first recognition network model is greater than the image resolution of the original part image, the recognition device can use an image signal processor to upsample the original part image according to the image resolution adapted to the first recognition network model, thereby obtaining the adjusted part image corresponding to the first recognition network model. When the image resolution adapted to the first recognition network model is less than the image resolution of the original part image, the recognition device can use an image signal processor to downsample the original part image according to the image resolution adapted to the first recognition network model, thereby obtaining the adjusted part image corresponding to the first recognition network model.

[0080] S104. Based on the above-mentioned adjustment part image, determine the preprocessed image of the above-mentioned adjustment part image, and transmit the preprocessed image of the above-mentioned adjustment part image to the central processing unit. The central processing unit is used to call the above-mentioned first recognition network model to perform part recognition on the preprocessed image of the above-mentioned adjustment part image to obtain the first object identifier corresponding to the above-mentioned adjustment part image. The first object identifier is used to reflect the object to which the above-mentioned recognized part belongs.

[0081] Specifically, based on the image of the adjusted part, histogram equalization is first applied to enhance image contrast, and then edge detection is performed using the Sobel operator. The image obtained after processing and detection is used as the preprocessed image of the adjusted part. The preprocessed image of the adjusted part is transmitted to the central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part to obtain the first object identifier corresponding to the adjusted part image.

[0082] In this application, the recognition device can determine the image of the adjusted part as a preprocessed image of the adjusted part, or preprocess the image of the adjusted part to obtain a preprocessed image of the adjusted part. The preprocessed image of the adjusted part is transmitted to a central processing unit (CPU), which calls the first recognition network model to perform part recognition on the preprocessed image of the adjusted part to obtain a first object identifier corresponding to the adjusted part image. That is, the CPU does not need to adjust the corresponding adjusted part image for the first recognition network model, thereby reducing the processing load on the CPU and improving the efficiency of part recognition.

[0083] The first object identifier can refer to the registration identifier of the object to which the identification part belongs in the identification application. The first object identifier can refer to an account, nickname, real name, etc.

[0084] It should be noted that the preprocessed image of the adjusted part can refer to the image of the adjusted part itself, or the preprocessed image of the adjusted part can refer to the image of the adjusted part obtained by preprocessing it through a hardware graphics accelerator. Here, preprocessing can refer to features extraction, feature dimensionality reduction, and other processing.

[0085] It should be noted that the central processing unit can be deployed in the identification device or in other devices. For example, the identification device can be a terminal, while other devices can refer to the server corresponding to the identification application.

[0086] In one embodiment, the central processing unit (CPU) is part of the identification device; the CPU includes at least two core identification components; that is, when the CPU is a multi-core CPU, the core identification component can refer to the computing engine within the CPU. The CPU is used to invoke the first identification network model to perform part identification on the preprocessed image of the adjusted part image, obtaining a first object identifier corresponding to the adjusted part image. This includes: the identification device can obtain the amount of tasks to be processed corresponding to the at least two core identification components from the task queues corresponding to each core identification component, and select the core identification component with the smallest amount of tasks to be processed from the at least two core identification components. Through the selected core identification component, the first identification network model is invoked to perform part identification on the preprocessed image of the adjusted part image, obtaining a first object identifier corresponding to the adjusted part image. By selecting the core identification component based on the amount of tasks to be processed, it is beneficial to achieve load balancing on a multi-core CPU, reasonably allocate tasks to each core identification component, avoid overloading a single core identification component, and improve the efficiency of part identification.

[0087] The task queue corresponding to the core recognition component includes tasks to be processed by the core recognition component, which are images to be recognized by the core recognition component. The number of tasks to be processed can refer to the number of images corresponding to the images to be recognized in the core recognition component.

[0088] In this application, an image signal processor dynamically determines a first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions based on the recognition performance parameters corresponding to each recognition network model. An adjustment region image adapted to the first recognition network model is then generated separately. This allows the first recognition network model to directly perform region recognition on the pre-processed image of the adjustment region, eliminating the need for the central processing unit to pre-process the original region image, thus reducing the processing time of the central processing unit and improving the efficiency of region recognition.

[0089] Further, please refer to Figure 4, which is a flowchart illustrating an image data processing method provided in an embodiment of this application. The method may include the following steps:

[0090] S201. Obtain the original part image including the recognition part, and the recognition performance parameters corresponding to N recognition network models respectively; N is a positive integer.

[0091] S202. Based on the recognition performance parameters corresponding to the above N recognition network models, determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models.

[0092] S203. If the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, then the image signal processor adjusts the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0093] In one feasible implementation, if the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, the image signal processor can resample the pixels of the original part image using the nearest neighbor interpolation method according to the image resolution adapted to the first recognition network model, thereby adjusting the resolution of the original part image and obtaining the adjusted part image corresponding to the first recognition network model.

[0094] In one embodiment, the original part image includes an infrared part image and a color part image captured for the identified part; adjusting the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain an adjusted part image corresponding to the first recognition network model includes: if the first recognition network model is a recognition network model for recognizing infrared part images, the recognition device can use an image signal processor to adjust the image resolution of the infrared part image according to the image resolution adapted to the first recognition network model to obtain an adjusted part image corresponding to the infrared part image. For ease of distinction, the adjusted part image corresponding to the infrared part image can be referred to as the first adjusted part image.

[0095] In one embodiment, if the first recognition network model is a recognition network model for recognizing colored parts of an image, the recognition device can use an image signal processor to adjust the image resolution of the colored part image according to the image resolution adapted to the first recognition network model, thereby obtaining an adjusted part image corresponding to the colored part image. For ease of distinction, the adjusted part image corresponding to the colored part image can be referred to as the second adjusted part image.

[0096] In one embodiment, if the first recognition network model includes a recognition network model a1 for recognizing infrared part images and a recognition network model a2 for recognizing color part images, the recognition device can use an image signal processor to adjust the image resolution of the infrared part image according to the image resolution adapted to the recognition network model a1, thereby obtaining a first adjusted part image corresponding to the infrared part image. Similarly, according to the image resolution adapted to the recognition network model a2, the image resolution of the color part image is adjusted to obtain a second adjusted part image corresponding to the color part image.

[0097] S204. Based on the above-mentioned image of the adjusted part, determine the preprocessed image of the adjusted part and transmit the preprocessed image of the adjusted part to the central processing unit.

[0098] The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain the first object identifier corresponding to the adjusted part image; the first object identifier is used to reflect the object to which the identified part belongs.

[0099] In one embodiment, the recognition device further includes a hardware graphics accelerator. The recognition device can use the hardware graphics accelerator to preprocess the image of the adjusted part according to the preprocessing network of the first recognition network model, obtaining a preprocessed image of the adjusted part. That is, the preprocessing process of the first recognition network model is moved forward to the hardware graphics accelerator, which reduces the computational load on the central processing unit (CPU), lowers the CPU's burden, and improves the efficiency of part recognition.

[0100] The preprocessing here may include at least one of feature extraction and feature dimensionality reduction.

[0101] In one embodiment, the preprocessing network of the first recognition network model includes a feature extraction layer and a feature dimensionality reduction layer. The process of preprocessing the adjusted part image using the hardware graphics accelerator and the preprocessing network of the first recognition network model to obtain a preprocessed image of the adjusted part includes: the recognition device can extract part features from the adjusted part image using the feature extraction layer of the first recognition network model via the hardware graphics accelerator to obtain a part feature map of the adjusted part image. This part feature map can include feature information in the adjusted part image that reflects the recognized part; the feature information can include at least one of texture, shape, and color. The recognition device can perform dimensionality reduction processing on the part feature map of the adjusted part image using the feature dimensionality reduction layer of the first recognition network model to obtain the preprocessed image of the adjusted part image. That is, dimensionality reduction processing can refer to extracting key feature information of the recognized part from the part feature map of the adjusted part image; this key feature information can refer to information associated with the recognition of the recognized part. Preprocessing the adjusted part image using a hardware graphics accelerator reduces the computational load on the central processing unit (CPU), lowers the CPU's burden, and improves the efficiency of part recognition.

[0102] In one embodiment, step S204 includes: the identification device generating a first driving signal based on the generation timestamp of the preprocessed image of the adjustment area image; the first driving signal is used to indicate that a readable image currently exists in the image signal processor, and the generation timestamp of the preprocessed image is used to reflect the generation time of the preprocessed image. The first driving signal is sent to the central processing unit (CPU); the CPU is further used to read the preprocessed image of the adjustment area image from the image signal processor based on the first driving signal. In other words, the image signal processor triggers the CPU to read the image via an interrupt-driven method, that is, when a readable image exists in the image signal processor, an interrupt signal (i.e., the first driving signal) is sent to the CPU. The CPU reads the image from the image signal processor via the interrupt signal, eliminating the need for the CPU to read the image in a polling manner. This significantly reduces the CPU's idle query time, lowers CPU resource consumption, and saves processing resources.

[0103] The drive signal is generated based on the generation timestamp of the preprocessed image of the part image (including the original part image and the adjusted part image). It is used to indicate that a readable image currently exists in the image signal processor. When a readable image exists in the image signal processor, the corresponding drive signal is sent to the central processing unit (CPU). The CPU then reads the corresponding preprocessed image from the image signal processor based on this drive signal. This interrupt-driven approach can significantly reduce the CPU's idle polling time and lower CPU resource consumption.

[0104] It should be noted that the generation timestamp of the preprocessed image of the adjusted part image can refer to the generation timestamp of the original part image. The generation timestamp of the original part image can be determined based on the exposure time of the camera when the original part image was taken. The camera can be a module in the recognition device, or the camera can refer to an independent device.

[0105] Among them, the polling method can refer to the central processing unit needing to periodically check whether there is a readable image in the image signal processor; the interrupt-driven method can refer to the central processing unit reading the image from the image signal processor only after receiving an interrupt signal.

[0106] In one embodiment, the central processing unit is used to invoke the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain the first object identifier corresponding to the adjusted part image. This includes: the recognition device can input the preprocessed image of the adjusted part image into the first recognition network model through the central processing unit, and perform part recognition on the preprocessed image of the adjusted part image through the first recognition network model to obtain the first object identifier corresponding to the adjusted part image.

[0107] S205. If N is greater than 1, and the above N recognition network models also include a second recognition network model whose recognition performance parameters meet the processor resource usage conditions, then the preprocessed image of the above original part image is transmitted to the above central processing unit.

[0108] In this application, when N is greater than 1, and the N recognition network models also include a second recognition network model whose recognition performance parameters meet the processor resource usage conditions, it indicates that the second recognition network model occupies relatively few processing resources, or that the processing latency corresponding to the second recognition network model is relatively small. This means that the second recognition network model is less prone to stuttering or other problems during part recognition. Therefore, the recognition device can transmit the preprocessed image of the original part image to the aforementioned central processing unit. This eliminates the need for the image signal processor to adjust the second recognition network model to obtain the appropriate adjusted part image; the preprocessed image corresponding to the original part image is directly sent to the central processing unit, thus reducing the processing load on the image signal processor.

[0109] The central processing unit is also used to call the second recognition network model to perform part recognition on the preprocessed image of the original part image, obtain the second object identifier corresponding to the original part image, and determine the recognition object identifier corresponding to the recognition part based on the first object identifier and the second object identifier corresponding to the original part image.

[0110] The preprocessed image of the original part image can refer to the original part image itself, or the preprocessed image of the original part image can be obtained by preprocessing the original part image using a hardware graphics accelerator.

[0111] In one embodiment, a similarity score can be calculated between the first object identifier and the second object identifier corresponding to the original part image, using a cosine similarity algorithm. A similarity threshold is set. If the similarity score is greater than the threshold, the object identifier with the higher confidence level between the first and second object identifiers is selected as the object identifier corresponding to the identified part. If the similarity score is less than or equal to the threshold, a more suitable object identifier is selected based on historical recognition data and the characteristics of the current recognition scene.

[0112] In one embodiment, transmitting the preprocessed image of the original part image to the central processing unit (CPU) includes: the recognition device generating a second driving signal based on the generation timestamp of the preprocessed image of the original part image; the generation timestamp reflects the generation time of the preprocessed image of the original part image, and the second driving signal reflects the presence of a readable image in the image signal processor. The second driving signal can be sent to the CPU; the CPU is further configured to read the preprocessed image of the original part image from the image signal processor based on the second driving signal. The image signal processor triggers the CPU to read the image via an interrupt-driven method, eliminating the need for the CPU to read the image in a polling manner. This significantly reduces the CPU's idle query time, lowers CPU resource consumption, and saves processing resources.

[0113] In one embodiment, the original part image may include an infrared part image and a color part image captured by the same identification part. The identification device can determine the generation timestamp of the infrared part image based on the exposure time of the infrared part image captured by the infrared camera, and generate a driving signal for the infrared part image based on the generation timestamp. Similarly, the generation timestamp of the color part image is determined based on the exposure time of the color part image captured by the color camera, and a driving signal for the color part image is generated based on the generation timestamp. The driving signals for the infrared and color parts images are sent to the central processing unit (CPU). The CPU can read a preprocessed image of the infrared part image and a preprocessed image of the adjusted part image obtained from the infrared part image from the image signal processor based on the driving signal of the infrared part image. Likewise, the CPU can read a preprocessed image of the color part image and a preprocessed image of the adjusted part image obtained from the color part image from the image signal processor based on the driving signal of the color part image. This significantly reduces the CPU's idle query time, lowers CPU resource consumption, and saves processing resources.

[0114] For example, Figure 5 shows the exposure timing diagram of a color camera. The frame time of the color camera is 40ms. Frame time refers to the time interval between capturing images of the identified area; that is, the color camera captures one frame of color image of the identified area every 40ms. Pulse 1 in Figure 5 reflects the exposure time and duration of the color camera. The exposure time of the color camera includes 35ms, 75ms, and 115ms, with an exposure duration of 5ms each. The color camera captures one frame of color image at each exposure time; that is, the color camera captures the first, second, and third frames of color image at 35ms, 75ms, and 115ms, respectively. Pulse 2 reflects the driving signal of the color image and the trigger reading time of the preprocessed image associated with the color image. Pulse 3 reflects the actual reading time of the preprocessed image associated with the color image by the central processing unit. For example, the actual reading time of the preprocessed image associated with the first frame of color part image is 40ms, the actual reading time of the preprocessed image associated with the second frame of color part image is 80ms, and the actual reading time of the preprocessed image associated with the third frame of color part image is 110ms.

[0115] The preprocessed image associated with the color part image may include at least one of the following: a preprocessed image of the color part image and a preprocessed image corresponding to the adjusted part image obtained based on the color part image.

[0116] For example, Figure 6 shows the exposure timing diagram of an infrared camera. The frame rate of the infrared camera is 40ms. The infrared camera captures a frame of the identified area every 40ms to obtain an infrared area image. Pulse 3 in Figure 6 reflects the exposure time and duration of the infrared camera. The exposure time of the infrared camera includes 38ms, 78ms, and 118ms, and the exposure duration is 5ms for each time. The infrared camera captures a frame of the infrared area image at each exposure time, that is, the infrared camera captures the first frame, the second frame, and the third frame of the infrared area image at 38ms, 78ms, and 118ms, respectively. Pulse 5 can reflect the driving signal of the infrared area image, the trigger reading time of the preprocessed image associated with the infrared area image, and the actual reading time of the preprocessed image associated with the infrared area image by the central processing unit. For example, the actual reading time of the preprocessed image associated with the first infrared region image is 40ms, the actual reading time of the preprocessed image associated with the second infrared region image is 80ms, and the actual reading time of the preprocessed image associated with the third infrared region image is 110ms.

[0117] The preprocessed image associated with the infrared region image may include at least one of the following: a preprocessed image of the infrared region image and a preprocessed image corresponding to the adjusted region image obtained based on the infrared region image.

[0118] In one embodiment, the recognition device can preprocess the original part image using a hardware graphics accelerator based on a preprocessing network associated with the original part image, obtaining a preprocessed image of the original part image. Preprocessing the original part image using a hardware graphics accelerator reduces the computational load on the central processing unit (CPU), thereby improving the efficiency of part recognition.

[0119] Among them, the preprocessing network associated with the original part image can refer to a preprocessing network that can adapt to the same image resolution as the original part image.

[0120] In one embodiment, the central processing unit is further configured to determine the frame synchronization relationship between the preprocessed image of the original part image and the preprocessed image of the adjusted part image based on the generation timestamps corresponding to the preprocessed image of the original part image and the preprocessed image of the adjusted part image, and to determine the identification object identifier corresponding to the identification part based on the first object identifier, the second object identifier corresponding to the original part image and the frame synchronization relationship, so as to ensure that the identification result optimized based on this application is accurate.

[0121] Specifically, when the time difference between the generation timestamp of the preprocessed image of the original part image and the generation timestamp of the preprocessed image of the adjusted part image is less than a difference threshold, it is determined that the preprocessed images of the original part image and the preprocessed images of the adjusted part image have a frame synchronization relationship, that is, the preprocessed images of the original part image and the preprocessed images of the adjusted part image include the same identification part. When the difference is greater than or equal to the difference threshold, it is determined that the preprocessed images of the original part image and the preprocessed images of the adjusted part image do not have a frame synchronization relationship, that is, the preprocessed images of the original part image and the preprocessed images of the adjusted part image do not include the same identification part. When the preprocessed images of the original part image and the preprocessed images of the adjusted part image have a frame synchronization relationship, the identification object identifier corresponding to the identification part is determined according to the first object identifier and the second object identifier corresponding to the original part image.

[0122] For example, the time difference Δt = |t1-t2| between the generation timestamp t1 of the preprocessed image of the original part image and the generation timestamp t2 of the preprocessed image of the adjusted part image can be calculated, and a time difference threshold Δt can be set. threshold If Δt < Δt threshold Then, it is determined that the preprocessed image of the original part image and the preprocessed image of the adjusted part image have a frame synchronization relationship; if Δt ≥ Δt threshold If so, it is determined that the two do not have a frame synchronization relationship.

[0123] In one embodiment, the preprocessing network associated with the original part image includes a feature extraction layer and a feature dimensionality reduction layer. The aforementioned preprocessing of the original part image using a hardware graphics accelerator, based on the preprocessing network associated with the original part image, to obtain a preprocessed image of the original part image, includes: the recognition device can use the aforementioned hardware graphics accelerator to extract part features from the original part image based on the feature extraction layer associated with the original part image, obtaining a part feature map of the original part image; the part feature map may include feature information in the original part image that reflects the recognition part, and the feature information may include at least one of texture, shape, color, etc. The recognition device can use the feature dimensionality reduction layer associated with the original part image to perform dimensionality reduction processing on the part feature map of the original part image, obtaining the preprocessed image of the original part image. That is, dimensionality reduction processing can refer to extracting key feature information of the recognition part from the part feature map of the original part image, and the preprocessed image of the original part image includes the key feature information of the recognition part extracted from the original part image, which may refer to information associated with the recognition of the recognition part. By using a hardware graphics accelerator to preprocess the original part images, the computational load on the central processing unit (CPU) can be reduced, thus improving the efficiency of part recognition.

[0124] In one embodiment, the central processing unit (CPU) is part of the identification device. The CPU is further configured to perform part identification on the preprocessed image of the original part image according to the second identification network model to obtain a second object identifier corresponding to the original part image. This includes: if the image resolution adapted to the second identification network model is different from the image resolution of the original part image, the CPU adjusts the image resolution of the preprocessed image of the original part image according to the image resolution adapted to the second identification network model to obtain an adjusted preprocessed image. The CPU inputs the adjusted preprocessed image into the second identification network model, performs part identification on the adjusted preprocessed image, and outputs the second object identifier corresponding to the original part image. If the image resolution adapted to the second identification network model is the same as the image resolution of the original part image, the CPU inputs the preprocessed image of the original part image into the second identification network model, performs part identification on the preprocessed image, and outputs the second object identifier corresponding to the original part image. By using the second recognition network model to perform part recognition on the preprocessed image of the original part image, it is possible to determine the identification object corresponding to the identified part through multiple recognition network models, which helps to improve the accuracy of part recognition.

[0125] It should be noted that since both the first object identifier and the second object identifier are used to reflect the object corresponding to the identified part, they can be the same. However, because different recognition network models have different recognition accuracies, the first object identifier and the second object identifier can also be different.

[0126] Specifically, when the first object identifier and the second object identifier are different, determining the identification object identifier corresponding to the identified part based on the first object identifier and the second object identifier corresponding to the original part image includes: determining the identification object identifier corresponding to the object to which the identified part belongs based on the probability corresponding to the first object identifier and the probability corresponding to the second object identifier. For example, when the probability corresponding to the first object identifier is greater than the probability corresponding to the second object identifier, the first object identifier can be determined as the identification object identifier of the identified part. When the probability corresponding to the first object identifier is less than the probability corresponding to the second object identifier, the second object identifier can be determined as the identification object identifier of the identified part.

[0127] The probability corresponding to the first object identifier can be output by the first recognition network model, which reflects the probability that the object identifier of the object to which the recognition part belongs is the first object identifier; the probability corresponding to the second object identifier can be output by the second recognition network model, which reflects the probability that the object identifier of the object to which the recognition part belongs is the second object identifier.

[0128] In one embodiment, when the first object identifier and the second object identifier are different, determining the object identifier corresponding to the identified part based on the first object identifier and the second object identifier corresponding to the original part image includes: the identification device weights the probability corresponding to the first object identifier based on the recognition accuracy of the first identification network model to obtain a first weighted probability p1; weights the probability corresponding to the second object identifier based on the recognition accuracy of the second identification network model to obtain a second weighted probability p2; and determines the object identifier of the identified part based on the first weighted probability p1 and the second weighted probability p2. For example, when the first weighted probability p1 is greater than the second weighted probability p2, the first object identifier can be determined as the object identifier of the identified part; when the first weighted probability p1 is less than the second weighted probability p2, the second object identifier can be determined as the object identifier of the identified part.

[0129] Weighted probability refers to the probability of identifying the object identifier corresponding to a given part of the image. This is determined by assigning a weight to different types of images (e.g., infrared and color images) based on their environmental parameters, and then multiplying this weight by the probability of the corresponding object identifier. By comparing the weighted probabilities of different object identifiers, the object identifier corresponding to the identified part can be more accurately determined, improving the accuracy of part identification. Summation probability refers to the probability of summing the weighted probabilities of two identical object identifiers when there are three common object identifiers: the first object identifier, the second object identifier corresponding to the color image, and the second object identifier corresponding to the infrared image. When determining the object identifier corresponding to a part of the image, this summed probability is compared with other weighted probabilities, and the object identifier with the highest probability is selected as the object identifier for the identified part.

[0130] In one embodiment, when the first object identifier is the same as the second object identifier, the above-mentioned determination of the identification object identifier corresponding to the identification part based on the first object identifier and the second object identifier corresponding to the original part image includes: the identification device can determine the first object identifier as the identification object identifier corresponding to the identification part.

[0131] It should be noted that the recognition performance parameters of the recognition network model satisfying the processor resource usage condition can refer to one or more of the following: 1. The processing resources used by the recognition network model in the part recognition process are less than the resource threshold; 2. The recognition network model is not among the top K recognition network models with the most processing resources among the N recognition network models; 3. The processing latency of the recognition network model in the part recognition process is less than the latency threshold; 4. The recognition network model is not among the top M recognition network models with the longest corresponding processing latency among the N recognition network models.

[0132] In one embodiment, the original part image includes an infrared part image and a color part image captured for the identified part. The first recognition network model is a recognition network model for recognizing the infrared part image. The recognition device can send the pre-processed images corresponding to the first adjusted part image, the infrared part image, and the color part image to the central processing unit. The central processing unit calls the first recognition network model to perform part recognition on the pre-processed image of the first adjusted part image, obtaining a first object identifier corresponding to the first adjusted part image. Then, through the second recognition network model b1, it performs part recognition on the pre-processed image corresponding to the infrared part image, obtaining a second object identifier corresponding to the infrared part image. Similarly, through the second recognition network model b2, it performs part recognition on the pre-processed image corresponding to the color part image, obtaining a second object identifier corresponding to the color part image. Based on the second object identifier corresponding to the color part image, the second object identifier corresponding to the infrared part image, and the first object identifier corresponding to the first adjusted part image, the identification object identifier corresponding to the identified part is determined. By performing part recognition on multiple part images (i.e., color part images and infrared part images), the accuracy of part recognition is improved.

[0133] For example, Figure 7 is a flowchart illustrating an image data processing method provided in an embodiment of this application. When the image resolution of both the infrared part image and the color part image is 720*988, assuming the first recognition network model is a recognition network model used to recognize the infrared part image, and the image resolution adapted to the first recognition network model is 128*128, the image signal processor can adjust the image resolution of the infrared part image according to the image resolution adapted to the first recognition network model to obtain an adjusted part image, i.e., the resolution of the adjusted part image is 128*128. The central processing unit can read the color data stream from the image signal processor through data callback 1, read the infrared data stream 1 from the image signal processor through data callback 2, and read the infrared data stream 2 from the image signal processor through data callback 3. The color data stream includes the preprocessed image corresponding to the color part image, the infrared data stream 1 includes the preprocessed image corresponding to the infrared part image, and the infrared data stream 2 includes the preprocessed image corresponding to the adjusted part image. The central processing unit (CPU) can perform frame synchronization processing on the preprocessed images corresponding to the color part image, the preprocessed image corresponding to the adjustment part image, and the preprocessed image corresponding to the infrared part image, based on the generation timestamps of the preprocessed images corresponding to the color part image, the infrared part image, and the adjustment part image. When there is a frame synchronization relationship between the preprocessed images corresponding to the color part image, the preprocessed images corresponding to the infrared part image, and the preprocessed images corresponding to the adjustment part image, the first recognition network model identifies the preprocessed image corresponding to the adjustment part image to obtain a first object identifier. The second recognition network model b1 identifies the part of the preprocessed image corresponding to the infrared part image to obtain a second object identifier. The second recognition network model b2 identifies the part of the preprocessed image corresponding to the color part image to obtain a second object identifier. Based on the second object identifier corresponding to the color part image, the second object identifier corresponding to the infrared part image, and the first object identifier corresponding to the first adjustment part image, the identification object identifier corresponding to the identified part is determined.

[0134] It should be noted that when the image resolution adapted to the second recognition network model b1 is 720*988, the second recognition network model b1 can directly perform part recognition on the preprocessed image corresponding to the infrared part image to obtain the second object identifier corresponding to the infrared part image. When the image resolution adapted to the second recognition network model b1 is not 720*988, the resolution of the preprocessed image corresponding to the infrared part image can be adjusted according to the image resolution adapted to the second recognition network model b1 to obtain the adjusted preprocessed image. Then, the second recognition network model b1 can perform part recognition on the adjusted preprocessed image to obtain the second object identifier corresponding to the infrared part image. Similarly, when the image resolution adapted to the second recognition network model b2 is 720*988, the second recognition network model b2 can directly perform part recognition on the preprocessed image corresponding to the color part image to obtain the second object identifier corresponding to the color part image. When the image resolution adapted to the second recognition network model b2 is not 720*988, the resolution of the preprocessed image corresponding to the color part image can be adjusted according to the image resolution adapted to the second recognition network model b2 to obtain the adjusted preprocessed image. The second object identifier corresponding to the color part image is obtained by performing part recognition on the adjusted preprocessed image through the second recognition network model b2.

[0135] Among them, data callback refers to the function used to read images. During image data processing, the central processing unit can call these functions to read the corresponding preprocessed images, such as color part images, infrared part images, and adjusted part images, from the image signal processor for subsequent part recognition operations.

[0136] In one embodiment, determining the identification object identifier corresponding to the identification region based on the first object identifier (i.e., the first object identifier corresponding to the first adjustment region image), the second object identifier corresponding to the color region image, and the second object identifier corresponding to the infrared region image includes: the identification device can determine a first weight corresponding to the infrared region image and a second weight corresponding to the color region image based on the shooting environment parameters of the identification region. Based on the first weight, the probability corresponding to the first object identifier is weighted to obtain a first weighted probability, i.e., the first weight is multiplied by the probability corresponding to the first object identifier. Based on the first weight, the probability of the second object identifier corresponding to the infrared region image is weighted to obtain a second weighted probability, i.e., the first weight is multiplied by the probability of the second object identifier corresponding to the infrared region image. Based on the second weight, the probability of the second object identifier corresponding to the color region image is weighted to obtain a third weighted probability, i.e., the second weight is multiplied by the probability of the second object identifier corresponding to the color region image. The recognition device can determine the object identifier corresponding to the recognition part based on the first weighted probability, the second weighted probability, the third weighted probability, the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image. By dynamically determining the object identifier of the recognition part through the shooting environment parameters of the recognition part, the accuracy of part recognition can be improved.

[0137] Shooting environment parameters refer to the relevant parameters of the environment in which the recognition area is located when it is photographed, mainly including light intensity and degree of occlusion. These parameters affect the quality and recognition effect of different types of area images (such as infrared area images and color area images). Based on the shooting environment parameters, corresponding weights can be determined for infrared area images and color area images respectively. Then, when determining the recognition object identifier corresponding to the recognition area, the accuracy of area recognition can be improved by weighting the probabilities corresponding to different object identifiers.

[0138] For example, when the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image are all different, the object identifier with the highest weighted probability can be used as the object identifier for the identified part. When the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image are all the same, the first object identifier can be used as the object identifier for the identified part. When there are two identical object identifiers among the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image, the weighted probabilities corresponding to the two identical object identifiers can be summed to obtain a summed probability. The object identifier corresponding to the highest probability among the summed probability and the weighted probability can be used as the object identifier for the identified part. For example, when the first object identifier is the same as the second object identifier corresponding to the color part image, but the second object identifier corresponding to the infrared part image is different from the first object identifier, the first weighted probability and the third weighted probability corresponding to the first object identifier can be summed to obtain the summed probability. If the summed probability is greater than or equal to the second weighted probability, the first object identifier can be determined as the identification object identifier of the identification part; if the summed probability is less than the second weighted probability, the second object identifier corresponding to the infrared part image can be determined as the identification object identifier of the identification part.

[0139] It should be noted that the environmental parameters for capturing the identified part can include light intensity and occlusion level. In situations with weak light intensity and high occlusion, infrared images can more clearly reveal the identified part. Therefore, taking light intensity as an example, when the light intensity is greater than a threshold, the first weight corresponding to the infrared part image is less than the second weight corresponding to the color part image; this is beneficial because the identification result of the color part image pair is prioritized during part recognition. When the light intensity is less than or equal to the threshold, the first weight corresponding to the infrared part image is greater than the second weight corresponding to the color part image; this is also beneficial because the identification result of the infrared part image pair is prioritized during part recognition. Dynamically determining the first and second weights based on the environmental parameters avoids the problem of low accuracy in part recognition under conditions of weak light intensity and high occlusion, thus improving the accuracy of part recognition.

[0140] For example, let the shooting environment parameter be light intensity I, and set the light intensity threshold I. threshold . When I>I threshold When the color region image corresponds to the second weight w2, it can be expressed as: The first weight w1 corresponding to the infrared region image is 1-w2; when I≤I threshold At that time, the first weight corresponding to the infrared region image The second weight w2 corresponding to the colored region image is 1 - w1. Where I min I represents the minimum light intensity. max This represents the maximum value of the light intensity.

[0141] Wherein, the second recognition network model b1 can refer to the network model among N recognition network models whose recognition performance parameters meet the processor resource usage conditions and are used to recognize infrared part images, and the second recognition network model b2 can refer to the network model among N recognition network models whose recognition performance parameters meet the processor resource usage conditions and are used to recognize color part images.

[0142] In one embodiment, the original part image includes an infrared part image and a color part image captured for the identified part, and the first recognition network model is a recognition network model for recognizing the color part image. The recognition device can send the pre-processed images corresponding to the second adjusted part image, the infrared part image, and the color part image to the central processing unit. The central processing unit is used to invoke the first recognition network model to perform part recognition on the pre-processed image of the second adjusted part image to obtain a first object identifier corresponding to the second adjusted part image. Through the second recognition network model b1, part recognition is performed on the pre-processed image corresponding to the infrared part image to obtain a second object identifier corresponding to the infrared part image. Through the second recognition network model b2, part recognition is performed on the pre-processed image corresponding to the color part image to obtain a second object identifier corresponding to the color part image. Based on the second object identifier corresponding to the color part image, the second object identifier corresponding to the infrared part image, and the first object identifier corresponding to the second adjusted part image, the identification object identifier corresponding to the identified part is determined. By performing part recognition on multiple part images (i.e., color part images and infrared part images), the accuracy of part recognition is improved.

[0143] In one embodiment, the original part image includes an infrared part image and a color part image captured for the identified part. The first identification network model includes an identification network model a1 for identifying the infrared part image and an identification network model a2 for identifying the color part image. The identification device can send the pre-processed images corresponding to the first adjustment part image, the second adjustment part image, the infrared part image, and the color part image to a central processing unit. The central processing unit is used to invoke the identification network model a1 to perform part identification on the pre-processed image of the first adjustment part image to obtain a first object identifier corresponding to the first adjustment part image; and to invoke the identification network model a2 to perform part identification on the pre-processed image of the second adjustment part image to obtain a first object identifier corresponding to the second adjustment part image. The second identification network model b1 is used to perform part identification on the pre-processed image corresponding to the infrared part image to obtain a second object identifier corresponding to the infrared part image. The second identification network model b2 is used to perform part identification on the pre-processed image corresponding to the color part image to obtain a second object identifier corresponding to the color part image. The identification object identifier corresponding to the identified part is determined based on the second object identifier corresponding to the color part image, the second object identifier corresponding to the infrared part image, the first object identifier corresponding to the second adjusted part image, and the first object identifier corresponding to the first adjusted part image. By performing part identification on multiple part images (i.e., color part images and infrared part images), the accuracy of part identification is improved.

[0144] In this application, when there is a second recognition network model among the N recognition network models whose recognition performance parameters meet the processor resource usage conditions, the preprocessed image of the original part image can be sent to the central processing unit, and the central processing unit will perform scaling processing (i.e. resolution adjustment) on the preprocessed image of the original part image. This helps to reduce the processing pressure of the image signal processor, avoid the image signal processor from experiencing lag, and improve the efficiency of part recognition.

[0145] The image data processing method of this application can be applied to payment scenarios, access control scenarios, and work attendance scenarios, as shown in Figure 8. Taking the application of the method of this application to a payment scenario as an example, the application is identified as a payment application, and the identification part is the palm. The identification device can be the payment application server 83a corresponding to the payment application. This payment scenario also includes the user terminal 82a corresponding to the user 81a and the payment terminal 84a used by the merchant for cashiering.

[0146] User terminal 82a can log in to the payment application based on the object identifier of user 81a to interact with the payment application server 83a. Payment terminal 84a has the payment application installed, and can log in to the payment application based on the object identifier of the merchant to interact with the payment application server 83a.

[0147] The payment process based on the palm of your hand in payment scenarios includes the following steps:

[0148] S81. User terminal 82a captures a palm image 85a, that is, user terminal 82a captures a palm image 85a of user 81a's palm. The palm image 85a may include at least one of infrared palm image and color palm image.

[0149] S82, User terminal 82a sends a registration request to payment application server 83a. User terminal 82a can generate a registration request, which includes the object identifier of user 81a and palm image 85a.

[0150] S83. The payment application server 83a extracts palm image features. That is, the payment application server 83a can parse the registration request to obtain the object identifier of user 81a and palm image 85a. Through the recognition network model, the palm image 85a is extracted to obtain the palm image features of palm image 85a.

[0151] S84, the payment application server 83a binds the palm image features with the object identifier, that is, it stores the palm image features and the object identifier according to the binding relationship between the palm image features and the object identifier.

[0152] S85, the payment application server 83a can generate a registration success notification, which is used to indicate that user 81a has successfully registered, that is, to indicate that user 81a can complete the payment operation in the payment application based on the palm of the hand.

[0153] S86. The payment application server 83a returns a registration success notification to the user terminal 82, and the user terminal 82a can display the registration success notification in the payment application.

[0154] S87. The payment terminal 84a captures a palm image to obtain an original palm image 86a. That is, when user 81a needs to perform a payment operation on the payment application, the payment terminal 84a captures a palm image of user 81a to obtain an original palm image 86a. The original palm image 86a may include at least one of an infrared palm image and a color palm image.

[0155] S88, Payment terminal 84a sends a payment request to payment application server 83a. That is, payment terminal 84a can generate a payment request and send it to payment application server 83a; the payment request includes the original palm image 86a, the quantity of the electronic resource to be paid, the merchant's account address, etc.

[0156] S89. Payment application server 83a performs palm recognition processing. Specifically, payment application server 83a can obtain the recognition performance parameters of the recognition network model and filter out the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions. When the image resolution adapted to the first recognition network model is different from the image resolution of the original palm image 86a, the image resolution of the original palm image 86a can be adjusted according to the image resolution adapted to the first recognition network model, resulting in an adjusted palm image (i.e., an adjusted portion image) of the original palm image 86a. This preprocessed image of the adjusted palm image is then sent to the central processing unit. The central processing unit calls the first recognition network model to compare the palm image features in the preprocessed image corresponding to the adjusted palm image with the palm image features stored in payment application server 83a.

[0157] S90. Payment application server 83a executes the payment. Specifically, if a specific palm image feature in payment application server 83a matches the palm image feature in the preprocessed image corresponding to the adjusted palm image, the object identifier corresponding to the specific palm image feature can be determined as the object identifier corresponding to the original palm image 86a. Based on the object identifier corresponding to the original palm image 86a, the account address of user 81a is determined. According to the payment request, electronic resources are transferred from user 81a's account address to the merchant's account address to complete the payment operation.

[0158] S91. Payment application server 83a generates a payment success notification. After the payment application server 83a completes the payment operation, it can generate a payment success notification to indicate to user 81a that the payment operation has been completed.

[0159] S92, The payment application server 83a returns a payment success notification to the payment terminal 84a, and the payment terminal 84a can display the payment success notification in the payment application.

[0160] In summary, in payment application scenarios, the image signal processor in the payment application server generates a pre-processed image corresponding to the palm image separately for the first recognition network model. This eliminates the need for the central processing unit (CPU) to pre-process the palm image, reducing CPU processing time and improving palm recognition efficiency. This also reduces the time required for palm verification devices (i.e., payment applications) and enhances the palm verification experience for low-cost devices.

[0161] Please refer to Figure 9, which is a schematic diagram of the structure of an image data processing apparatus provided in an embodiment of this application. As shown in Figure 9, the image data processing apparatus may include:

[0162] The acquisition module 911 is used to acquire the original part image including the recognition part, and the recognition performance parameters corresponding to N recognition network models respectively; N is a positive integer;

[0163] The determination module 912 is used to determine, based on the recognition performance parameters corresponding to the above N recognition network models, the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models;

[0164] The adjustment module 913 is used to adjust the image resolution of the original part image according to the image resolution adapted by the first recognition network model if the image resolution adapted by the first recognition network model is different from the image resolution of the original part image, so as to obtain the adjusted part image corresponding to the first recognition network model.

[0165] The transmission module 914 is used to determine a preprocessed image of the adjusted part image based on the adjusted part image, and transmit the preprocessed image of the adjusted part image to the central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain a first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

[0166] Optionally, the transmission module 914 is also used for:

[0167] If N is greater than 1, and the above N recognition network models also include a second recognition network model whose recognition performance parameters meet the processor resource usage conditions, then the preprocessed image of the above original part image is transmitted to the above central processing unit.

[0168] The central processing unit is also used to call the second recognition network model to perform part recognition on the preprocessed image of the original part image, obtain the second object identifier corresponding to the original part image, and determine the recognition object identifier corresponding to the recognition part based on the first object identifier and the second object identifier corresponding to the original part image.

[0169] Optionally, the transmission module 914 is specifically used to generate a first driving signal based on the generation timestamp of the preprocessed image of the image of the adjusted part.

[0170] The first driving signal is sent to the central processing unit; the central processing unit is also used to read the preprocessed image of the adjusted part image from the image signal processor based on the first driving signal.

[0171] A second driving signal is generated based on the generation timestamp of the preprocessed image of the original part image mentioned above;

[0172] The second driving signal is sent to the central processing unit; the central processing unit is also used to read the preprocessed image of the original part image from the image signal processor based on the second driving signal.

[0173] Optionally, the above-mentioned identification device also includes a hardware graphics accelerator, and the device also includes a processing module 915;

[0174] The transmission module 914 is specifically used to preprocess the image of the adjusted part through the hardware graphics accelerator and according to the preprocessing network of the first recognition network model to obtain the preprocessed image of the adjusted part.

[0175] The processing module 915 is used to preprocess the original part image according to the preprocessing network associated with the original part image to obtain a preprocessed image of the original part image.

[0176] Optionally, the preprocessing network of the first recognition network model mentioned above includes a feature extraction layer and a feature dimensionality reduction layer;

[0177] Transmission module 914 is specifically used for:

[0178] Using the aforementioned hardware graphics accelerator, based on the feature extraction layer of the aforementioned first recognition network model, the aforementioned adjustment region image is subjected to region feature extraction to obtain the region feature map of the aforementioned adjustment region image.

[0179] Based on the feature reduction layer of the first recognition network model, the feature map of the adjusted part image is reduced in dimension to obtain the preprocessed image of the adjusted part image.

[0180] Optionally, the aforementioned central processing unit belongs to the aforementioned identification device; the processing module 915 is further used for:

[0181] If the image resolution adapted by the second recognition network model is different from the image resolution of the original part image, then the central processing unit adjusts the image resolution of the preprocessed image of the original part image according to the image resolution adapted by the second recognition network model to obtain the adjusted preprocessed image.

[0182] The second recognition network model is invoked to perform part recognition on the adjusted preprocessed image to obtain the second object identifier corresponding to the original part image.

[0183] Optionally, the original part image mentioned above includes an infrared part image and a color part image obtained by capturing images of the identified part;

[0184] The adjustment module 913 is specifically used to adjust the image resolution of the infrared part image according to the image resolution adapted to the first recognition network model if the first recognition network model is a recognition network model for recognizing infrared part images, so as to obtain the adjusted part image corresponding to the first recognition network model.

[0185] The determining module 912 is further configured to determine the identification object identifier corresponding to the identification part based on the first object identifier, the second object identifier corresponding to the color part image and the second object identifier corresponding to the infrared part image.

[0186] Optionally, the determining module 912 is specifically used to determine the first weight corresponding to the infrared part image and the second weight corresponding to the color part image based on the shooting environment parameters of the above-mentioned identification part.

[0187] Based on the first weight, the probability corresponding to the first object identifier is weighted to obtain the first weighted probability. Based on the first weight, the probability of the second object identifier corresponding to the infrared part image is weighted to obtain the second weighted probability.

[0188] Based on the second weight mentioned above, the probability of the second object identifier corresponding to the above colored part image is weighted to obtain the third weighted probability;

[0189] Based on the first weighted probability, the second weighted probability, the third weighted probability, the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image, the identification object identifier corresponding to the identification part is determined.

[0190] Optionally, the central processing unit is further configured to determine the frame synchronization relationship between the preprocessed image of the original part image and the preprocessed image of the adjusted part image based on the generation timestamps corresponding to the preprocessed image of the original part image and the preprocessed image of the adjusted part image, and to determine the identification object identifier corresponding to the identification part based on the first object identifier, the second object identifier corresponding to the original part image and the frame synchronization relationship.

[0191] Optionally, the aforementioned central processing unit belongs to the aforementioned identification device; the aforementioned central processing unit includes at least two core identification components;

[0192] The processing module 915 is also used to: obtain the amount of tasks to be processed corresponding to the above-mentioned at least two core identification components;

[0193] From the above at least two core identification components, select the core identification component with the smallest amount of pending tasks;

[0194] By selecting the core recognition component and calling the first recognition network model, the preprocessed image of the adjusted part image is used to identify the part, thereby obtaining the first object identifier corresponding to the adjusted part image.

[0195] Optionally, when N is greater than 1, the determining module 912 is specifically used to obtain the processing resources occupied by the above N recognition network models in the part recognition process from the recognition performance parameters corresponding to the above N recognition network models respectively.

[0196] Based on the processing resources corresponding to the above N recognition network models, select K recognition network models from the above N recognition network models; the processing resources corresponding to the above K recognition network models are all greater than the processing resources corresponding to the recognition network models that were not selected among the above N recognition network models, and K is a positive integer less than N;

[0197] The above K recognition network models are selected as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

[0198] Optionally, when N is greater than 1, the determining module 912 is specifically used to obtain the processing delay of the above N recognition network models in the part recognition process from the recognition performance parameters corresponding to the above N recognition network models respectively;

[0199] Based on the processing latency corresponding to the above N recognition network models, select M recognition network models from the above N recognition network models; the processing latency corresponding to the above M recognition network models is greater than the processing latency corresponding to the recognition network models that were not selected among the above N recognition network models, and M is a positive integer less than N;

[0200] The M recognition network models mentioned above are identified as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

[0201] Optionally, the adjustment module 913 is specifically used to determine the image size adapted to the first recognition network model based on the image resolution adapted to the first recognition network model through the image signal processor.

[0202] Based on the image size adapted to the first recognition network model, the image size of the original part image is adjusted to obtain the adjusted part image corresponding to the first recognition network model.

[0203] In this application, an image signal processor dynamically determines a first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions based on the recognition performance parameters corresponding to each recognition network model. An adjustment region image adapted to the first recognition network model is then generated separately. This allows the first recognition network model to directly perform region recognition on the pre-processed image of the adjustment region, eliminating the need for the central processing unit to pre-process the original region image, thus reducing the processing time of the central processing unit and improving the efficiency of region recognition.

[0204] Please refer to Figure 10, which is a schematic diagram of the structure of a computer device provided in an embodiment of this application. As shown in Figure 10, the computer device may refer to the aforementioned identification device, and the aforementioned computer device 1000 may refer to a terminal or server, including: a processor 1001, a network interface 1004, and a memory 1005. In addition, the aforementioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. In some embodiments, the user interface 1003 may include a display screen and a keyboard. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. As shown in Figure 10, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a computer program.

[0205] In the computer device 1000 shown in Figure 10, the network interface 1004 provides network communication functions; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the computer program stored in the memory 1005 to implement the following steps of this application:

[0206] Obtain the original image of the region to be identified, and the recognition performance parameters corresponding to N recognition network models; N is a positive integer.

[0207] Based on the recognition performance parameters corresponding to the above N recognition network models, determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the above N recognition network models;

[0208] If the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, then the image signal processor adjusts the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0209] Based on the image of the adjusted part, a preprocessed image of the adjusted part is determined, and the preprocessed image of the adjusted part is transmitted to the central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part to obtain the first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

[0210] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0211] If N is greater than 1, and the above N recognition network models also include a second recognition network model whose recognition performance parameters meet the processor resource usage conditions, then the preprocessed image of the above original part image is transmitted to the above central processing unit.

[0212] The central processing unit is also used to call the second recognition network model to perform part recognition on the preprocessed image of the original part image, obtain the second object identifier corresponding to the original part image, and determine the recognition object identifier corresponding to the recognition part based on the first object identifier and the second object identifier corresponding to the original part image.

[0213] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0214] A first driving signal is generated based on the generation timestamp of the preprocessed image of the image of the adjusted part.

[0215] The first driving signal is sent to the central processing unit; the central processing unit is also used to read the preprocessed image of the adjusted part image from the image signal processor based on the first driving signal.

[0216] The above-mentioned transmission of the preprocessed image of the original part image to the central processing unit includes:

[0217] A second driving signal is generated based on the generation timestamp of the preprocessed image of the original part image mentioned above;

[0218] The second driving signal is sent to the central processing unit; the central processing unit is also used to read the preprocessed image of the original part image from the image signal processor based on the second driving signal.

[0219] Optionally, the aforementioned identification device further includes a hardware graphics accelerator, and the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0220] Using the aforementioned hardware graphics accelerator, the image of the adjusted part is preprocessed according to the preprocessing network of the aforementioned first recognition network model to obtain the preprocessed image of the adjusted part.

[0221] Based on the preprocessing network associated with the original part image, the original part image is preprocessed to obtain a preprocessed image of the original part image.

[0222] Optionally, the preprocessing network of the first recognition network model mentioned above includes a feature extraction layer and a feature dimensionality reduction layer;

[0223] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0224] Using the aforementioned hardware graphics accelerator, based on the feature extraction layer of the aforementioned first recognition network model, the aforementioned adjustment region image is subjected to region feature extraction to obtain the region feature map of the aforementioned adjustment region image.

[0225] Based on the feature reduction layer of the first recognition network model, the feature map of the adjusted part image is reduced in dimension to obtain the preprocessed image of the adjusted part image.

[0226] Optionally, the aforementioned central processing unit belongs to the aforementioned identification device; optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0227] If the image resolution adapted by the second recognition network model is different from the image resolution of the original part image, then the central processing unit adjusts the image resolution of the preprocessed image of the original part image according to the image resolution adapted by the second recognition network model to obtain the adjusted preprocessed image.

[0228] The second recognition network model is invoked to perform part recognition on the adjusted preprocessed image to obtain the second object identifier corresponding to the original part image.

[0229] Optionally, the original part image mentioned above includes an infrared part image and a color part image obtained by capturing images of the identified part;

[0230] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0231] If the first recognition network model is a recognition network model for recognizing infrared part images, then the image signal processor adjusts the image resolution of the infrared part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model.

[0232] The above-mentioned determination of the identification object identifier corresponding to the identification region based on the first object identifier and the second object identifier corresponding to the original region image includes:

[0233] Based on the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image, the identification object identifier corresponding to the identification part is determined.

[0234] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0235] Based on the shooting environment parameters of the above-mentioned identification parts, the first weight corresponding to the above-mentioned infrared part image and the second weight corresponding to the above-mentioned color part image are determined;

[0236] Based on the first weight, the probability corresponding to the first object identifier is weighted to obtain the first weighted probability. Based on the first weight, the probability of the second object identifier corresponding to the infrared part image is weighted to obtain the second weighted probability.

[0237] Based on the second weight mentioned above, the probability of the second object identifier corresponding to the above colored part image is weighted to obtain the third weighted probability;

[0238] Based on the first weighted probability, the second weighted probability, the third weighted probability, the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image, the identification object identifier corresponding to the identification part is determined.

[0239] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0240] The aforementioned central processing unit is further configured to determine the frame synchronization relationship between the preprocessed image of the original part image and the preprocessed image of the adjusted part image based on the generation timestamps corresponding to the preprocessed image of the original part image and the preprocessed image of the adjusted part image, and to determine the identification object identifier corresponding to the identification part based on the first object identifier, the second object identifier corresponding to the original part image and the frame synchronization relationship.

[0241] Optionally, the aforementioned central processing unit belongs to the aforementioned identification device; the aforementioned central processing unit includes at least two core identification components;

[0242] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0243] Obtain the amount of tasks to be processed corresponding to at least two of the above core recognition components;

[0244] From the above at least two core identification components, select the core identification component with the smallest amount of pending tasks;

[0245] By selecting the core recognition component and calling the first recognition network model, the preprocessed image of the adjusted part image is used to identify the part, thereby obtaining the first object identifier corresponding to the adjusted part image.

[0246] Optionally, when N is greater than 1, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0247] From the recognition performance parameters corresponding to the above N recognition network models, obtain the processing resources occupied by each of the above N recognition network models in the process of part recognition;

[0248] Based on the processing resources corresponding to the above N recognition network models, select K recognition network models from the above N recognition network models; the processing resources corresponding to the above K recognition network models are all greater than the processing resources corresponding to the recognition network models that were not selected among the above N recognition network models, and K is a positive integer less than N;

[0249] The above K recognition network models are selected as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

[0250] Optionally, when N is greater than 1, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0251] From the recognition performance parameters corresponding to the above N recognition network models, obtain the processing delay of the above N recognition network models in the part recognition process;

[0252] Based on the processing latency corresponding to the above N recognition network models, select M recognition network models from the above N recognition network models; the processing latency corresponding to the above M recognition network models is greater than the processing latency corresponding to the recognition network models that were not selected among the above N recognition network models, and M is a positive integer less than N;

[0253] The M recognition network models mentioned above are identified as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

[0254] Optionally, the processor 1001 can be used to call a computer program stored in the memory 1005 to implement the following steps of this application:

[0255] The image signal processor described above determines the image size adapted to the first recognition network model based on the image resolution adapted to the first recognition network model.

[0256] Based on the image size adapted to the first recognition network model, the image size of the original part image is adjusted to obtain the adjusted part image corresponding to the first recognition network model.

[0257] In this application, an image signal processor dynamically determines a first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions based on the recognition performance parameters corresponding to each recognition network model. An adjustment region image adapted to the first recognition network model is then generated separately. This allows the first recognition network model to directly perform region recognition on the pre-processed image of the adjustment region, eliminating the need for the central processing unit to pre-process the original region image, thus reducing the processing time of the central processing unit and improving the efficiency of region recognition.

[0258] It should be understood that the computer device described in the embodiments of this application can execute the image data processing method described in the corresponding embodiments above, and can also execute the image data processing apparatus described in the corresponding embodiments above, which will not be repeated here. In addition, the beneficial effects of using the same method will not be repeated here either.

[0259] In practice, the collection and processing of data in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the data subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the data subject.

[0260] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned image data processing apparatus. This computer program includes program instructions, which, when executed by the processor, enable the execution of the image data processing method described in the corresponding embodiments above. Therefore, these descriptions will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application.

[0261] As an example, the above program instructions can be deployed and executed on a computer device, or deployed and executed on at least two computer devices in one location, or executed on at least two computer devices distributed in at least two locations and interconnected by a communication network. At least two computer devices distributed in at least two locations and interconnected by a communication network can form a blockchain network.

[0262] The aforementioned computer-readable storage medium can be an internal storage unit of the image data processing apparatus provided in any of the foregoing embodiments or the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card (SMC), secure digital (SD) card, flash memory card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0263] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish content in different media, rather than to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0264] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0265] In practice, the collection and processing of data in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the data subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the data subject.

[0266] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image data processing method described in the preceding embodiments; therefore, it will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiments of this application.

[0267] In summary, this application provides an image data processing method, apparatus, device, computer-readable storage medium, and computer program product. The recognition device acquires an original image of the identified region and recognition performance parameters of N recognition network models. Based on these parameters, it determines a first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions. Typically, the resolution of the original region image differs from the resolution adapted to the recognition network model. If the central processing unit (CPU) were to adjust the image resolution for each recognition network model, it would increase its processing burden and time consumption. However, by using an image signal processor to adjust the resolution of the original region image according to the resolution adapted to the first recognition network model, an adjusted region image is obtained. This avoids the CPU performing this operation, reduces the CPU's computational load on image resolution adjustment, lowers its processing pressure, and allows it to allocate more resources to calling the recognition network model for region recognition, thereby improving the efficiency of region recognition.

[0268] Furthermore, when N is greater than 1 and a second recognition network model exists whose recognition performance parameters satisfy the processor resource usage conditions, the recognition device transmits the preprocessed image of the original part image to the central processing unit (CPU). The CPU then calls the second recognition network model to perform part recognition to obtain the second object identifier, and combines it with the first object identifier to determine the object identifier corresponding to the recognized part. Since the second recognition network model consumes fewer processing resources and has a shorter processing latency, it does not require the image signal processor to adjust the image resolution. This reduces the processing pressure on the image signal processor, making the task allocation between the image signal processor and the CPU more reasonable, and improving the processing efficiency and resource utilization of the entire recognition system.

[0269] When transmitting preprocessed images of the adjusted and original parts, the recognition device generates a first drive signal and a second drive signal based on their generation timestamps, respectively. The central processing unit (CPU) then reads the image from the image signal processor (ESP) based on these signals. Traditional polling methods require the CPU to periodically check the ESP for readable images, resulting in significant idle query time and wasted resources. In contrast, the interrupt-driven method triggers the CPU to read images only when they are available, significantly reducing CPU idle query time, lowering CPU resource consumption, and improving the efficient utilization of system resources.

[0270] If the recognition device includes a hardware graphics accelerator, the hardware graphics accelerator preprocesses the image of the adjusted part according to the preprocessing network of the first recognition network model, and simultaneously preprocesses the image of the original part according to the preprocessing network associated with it. The hardware graphics accelerator has dedicated hardware circuitry and algorithms, enabling it to efficiently complete image preprocessing tasks such as feature extraction and feature dimensionality reduction. Shifting the preprocessing process from the central processing unit (CPU) to the hardware graphics accelerator reduces the computational load on the CPU, allowing it to focus more on the core recognition task and improving the efficiency and speed of part recognition.

[0271] When the preprocessing network of the first recognition network model includes a feature extraction layer and a feature dimensionality reduction layer, the hardware graphics accelerator first extracts the feature map of the adjusted part image through the feature extraction layer, and then performs dimensionality reduction processing through the feature dimensionality reduction layer to obtain the preprocessed image. The feature extraction layer can accurately extract the key feature information of the recognition part and remove redundant information, while the feature dimensionality reduction layer further reduces the amount of data, making the data transmitted to the central processing unit more concise. This not only reduces the bandwidth requirements for data transmission, but also reduces the computational load on the central processing unit when processing large amounts of data, improving recognition efficiency and accuracy.

[0272] When the central processing unit (CPU) is part of the recognition device and the image resolution adapted to the second recognition network model differs from that of the original part image, the CPU adjusts the resolution of the preprocessed image of the original part before calling the second recognition network model for recognition. This adjustment allows the second recognition network model to process the image better, as the adapted image resolution enables the recognition network model to extract image features more accurately, thereby improving recognition accuracy. Simultaneously, it makes efficient use of the CPU's computing resources, avoiding recognition errors and resource waste caused by image resolution mismatch.

[0273] If the original part image includes both infrared and color part images, when the first recognition network model is used to recognize the infrared part image, the recognition device adjusts the resolution of the infrared part image to obtain an adjusted part image, and then combines the first object identifier, the color part image, and the second object identifier corresponding to the infrared part image to determine the object identifier. Different types of images contain different feature information. Infrared part images can provide clearer features in low light or occluded conditions, while color part images can provide rich color and texture information. By comprehensively utilizing multiple images for recognition, features of the identified part can be obtained from multiple angles, improving the accuracy and reliability of the recognition.

[0274] When identifying object identifiers, the recognition device determines the weights of infrared and color images based on environmental parameters, and then weights the probabilities corresponding to each object identifier. The shooting environment affects the quality and recognition performance of different image types. For example, in well-lit environments, color images may perform better, while in low-light environments, infrared images are more advantageous. Dynamically adjusting the weights based on environmental parameters makes the recognition results more consistent with reality, improving accuracy and adaptability.

[0275] The central processing unit (CPU) determines the frame synchronization relationship based on the generation timestamps of the original and adjusted preprocessed images, and combines this with the first and second object identifiers to determine the identification object. Ensuring frame synchronization guarantees that images acquired at the same time point are used for recognition, avoiding recognition errors caused by image asynchrony. Because asynchronous images may contain identification part information from different times, interfering with the judgment of the recognition network model, frame synchronization processing improves the accuracy and reliability of the recognition results.

[0276] When the central processing unit (CPU) is part of the identification device and contains at least two core identification components, the CPU selects the component with the smallest workload for the identification task based on the workload of each component. Different core identification components of a multi-core CPU may have different processing capabilities and load conditions. If the task allocation is unreasonable, some components may be overloaded while others remain idle. Allocating tasks based on the workload achieves load balancing for the multi-core CPU, improving its overall processing efficiency and resource utilization.

[0277] When N is greater than 1, the recognition device selects the first recognition network model based on its processing resource consumption or processing latency. Identifying models that consume excessive processing resources or have excessive processing latency and adjusting their image resolution can reduce their resource consumption and processing latency during the recognition process. This is because these models consume more resources and time when processing images with mismatched resolutions; adjusting the image resolution allows them to operate more efficiently, improving overall processing efficiency and system stability.

[0278] The recognition device uses an image signal processor to determine the image size based on the image resolution adapted to the first recognition network model, and then adjusts the size of the original part image. This image size adjustment allows for a more accurate fit between the adjusted part image and the recognition network model, as image size is a crucial factor affecting the performance of the recognition network model. A suitable image size enables the recognition network model to extract image features more accurately, improving both the accuracy and efficiency of the recognition process.

[0279] Clearly defined processor resource usage conditions can accurately filter out recognition network models with different performance levels. Based on these conditions, tasks can be allocated rationally, allowing recognition network models with varying performance to function effectively within appropriate processing flows. For example, for the first recognition network model, which consumes a lot of processing resources and has a long processing latency, the image signal processor adjusts its image resolution; for the second recognition network model, which consumes fewer processing resources and has a shorter processing latency, preprocessed images of the original parts are used directly for recognition. This task allocation method improves the overall processing efficiency and resource utilization of the system, enhancing its stability and reliability.

[0280] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0281] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowcharts and / or structural diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable network-connected device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable network-connected device, create means for implementing the functions specified in one or more blocks of the flowcharts and / or one or more blocks of the structural diagrams. These computer program instructions can also be stored in a computer-readable storage medium capable of directing a computer or other programmable network-connected device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more blocks of the flowcharts and / or one or more blocks of the structural diagrams. These computer program instructions may also be loaded onto a computer or other programmable network-connected device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide steps for implementing the functions specified in one or more flowcharts and / or one or more structural diagrams in blocks.

[0282] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image data processing method applied to a recognition device, the recognition device including an image signal processor, the method comprising: Obtain the original part image including the identification part, and the recognition performance parameters corresponding to N recognition network models; N is a positive integer; Based on the recognition performance parameters corresponding to the N recognition network models, determine the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the N recognition network models; If the image resolution adapted to the first recognition network model is different from the image resolution of the original part image, then the image signal processor adjusts the image resolution of the original part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model. and Based on the image of the adjusted part, a preprocessed image of the adjusted part is determined, and the preprocessed image of the adjusted part is transmitted to a central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part to obtain a first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

2. The method of claim 1, further comprising: If N is greater than 1, and the N recognition network models also include a second recognition network model whose recognition performance parameters meet the processor resource usage conditions, then the preprocessed image of the original part image is transmitted to the central processing unit. The central processing unit is further configured to invoke the second recognition network model to perform part recognition on the preprocessed image of the original part image, obtain the second object identifier corresponding to the original part image, and determine the recognition object identifier corresponding to the recognized part based on the first object identifier and the second object identifier corresponding to the original part image.

3. The method as described in claim 2, wherein transmitting the preprocessed image of the adjusted area image to the central processing unit comprises: A first driving signal is generated based on the generation timestamp of the preprocessed image of the image of the adjusted part; The first drive signal is sent to the central processing unit; The central processing unit is also used to read a preprocessed image of the adjusted part image from the image signal processor based on the first driving signal; The step of transmitting the preprocessed image of the original part image to the central processing unit includes: A second driving signal is generated based on the generation timestamp of the preprocessed image of the original part image; The second drive signal is sent to the central processing unit; the central processing unit is also used to read the preprocessed image of the original part image from the image signal processor based on the second drive signal.

4. The method as described in claim 2 or 3, wherein the recognition device further comprises a hardware graphics accelerator, and the step of determining a preprocessed image of the adjusted part image based on the adjusted part image comprises: The hardware graphics accelerator is used to preprocess the image of the adjusted part according to the preprocessing network of the first recognition network model to obtain the preprocessed image of the adjusted part. The method further includes: The original part image is preprocessed according to the preprocessing network associated with it to obtain a preprocessed image of the original part image.

5. The method as described in claim 4, wherein the preprocessing network of the first recognition network model includes a feature extraction layer and a feature dimensionality reduction layer; The step of preprocessing the image of the adjusted part using the hardware graphics accelerator according to the preprocessing network of the first recognition network model to obtain a preprocessed image of the adjusted part includes: Using the hardware graphics accelerator, based on the feature extraction layer of the first recognition network model, the image of the adjusted part is subjected to part feature extraction to obtain the part feature map of the adjusted part image. Based on the feature reduction layer of the first recognition network model, the feature map of the adjusted part image is reduced in dimension to obtain the preprocessed image of the adjusted part image.

6. The method according to any one of claims 2 to 5, wherein the central processing unit belongs to the recognition device; the central processing unit is further configured to perform part recognition on the preprocessed image of the original part image according to the second recognition network model to obtain a second object identifier corresponding to the original part image, including: If the image resolution adapted to the second recognition network model is different from the image resolution of the original part image, then the central processing unit adjusts the image resolution of the preprocessed image of the original part image according to the image resolution adapted to the second recognition network model to obtain the adjusted preprocessed image. The second recognition network model is invoked to perform part recognition on the adjusted preprocessed image to obtain the second object identifier corresponding to the original part image.

7. The method according to any one of claims 2 to 6, wherein the original part image comprises an infrared part image and a color part image obtained by capturing images of the identified part; The step of adjusting the image resolution of the original part image using the image signal processor, based on the image resolution adapted to the first recognition network model, to obtain the adjusted part image corresponding to the first recognition network model, includes: If the first recognition network model is a recognition network model for recognizing infrared part images, then the image signal processor adjusts the image resolution of the infrared part image according to the image resolution adapted to the first recognition network model to obtain the adjusted part image corresponding to the first recognition network model. The step of determining the identification object identifier corresponding to the identification region based on the first object identifier and the second object identifier corresponding to the original region image includes: The identification object identifier corresponding to the identification part is determined based on the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image.

8. The method of claim 7, wherein determining the identification object identifier corresponding to the identification region based on the first object identifier, the second object identifier corresponding to the color region image, and the second object identifier corresponding to the infrared region image comprises: Based on the shooting environment parameters of the identified part, determine the first weight corresponding to the infrared part image and the second weight corresponding to the color part image; Based on the first weight, the probability corresponding to the first object identifier is weighted to obtain a first weighted probability; based on the first weight, the probability of the second object identifier corresponding to the infrared part image is weighted to obtain a second weighted probability. Based on the second weight, the probability of the second object identifier corresponding to the colored part image is weighted to obtain the third weighted probability; The identification object identifier corresponding to the identification part is determined based on the first weighted probability, the second weighted probability, the third weighted probability, the first object identifier, the second object identifier corresponding to the color part image, and the second object identifier corresponding to the infrared part image.

9. The method according to any one of claims 2 to 8, wherein the method further comprises: The central processing unit is further configured to determine the frame synchronization relationship between the preprocessed image of the original part image and the preprocessed image of the adjusted part image based on the generation timestamps corresponding to the preprocessed image of the original part image and the preprocessed image of the adjusted part image, and to determine the identification object identifier corresponding to the identification part based on the first object identifier, the second object identifier corresponding to the original part image and the frame synchronization relationship.

10. The method according to any one of claims 1 to 9, wherein the central processing unit belongs to the identification device; the central processing unit includes at least two core identification components; The central processing unit is used to perform part recognition on the preprocessed image of the adjusted part image according to the first recognition network model, to obtain a first object identifier corresponding to the adjusted part image, including: Obtain the amount of tasks to be processed corresponding to each of the at least two core identification components; From the at least two core identification components, select the core identification component with the smallest amount of tasks to be processed; By using the selected core recognition component, the first recognition network model is invoked to perform part recognition on the preprocessed image of the adjusted part image, thereby obtaining the first object identifier corresponding to the adjusted part image.

11. The method according to any one of claims 1 to 10, wherein when N is greater than 1, determining, based on the recognition performance parameters corresponding to the N recognition network models respectively, a first recognition network model whose recognition performance parameters do not meet the processor resource usage condition from the N recognition network models includes: From the recognition performance parameters corresponding to the N recognition network models, obtain the processing resources occupied by each of the N recognition network models in the part recognition process; Based on the processing resources corresponding to the N recognition network models, K recognition network models are selected from the N recognition network models; the processing resources corresponding to the K recognition network models are all greater than the processing resources corresponding to the recognition network models that were not selected from the N recognition network models, and K is a positive integer less than N; The K recognition network models are selected as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

12. The method according to any one of claims 1 to 11, wherein when N is greater than 1, determining, based on the recognition performance parameters corresponding to the N recognition network models respectively, a first recognition network model whose recognition performance parameters do not meet the processor resource usage condition from the N recognition network models includes: The processing delay of the N recognition network models in the part recognition process is obtained from the recognition performance parameters corresponding to the N recognition network models respectively. Based on the processing latency corresponding to the N recognition network models, M recognition network models are selected from the N recognition network models; the processing latency corresponding to each of the M recognition network models is greater than the processing latency corresponding to the recognition network models that were not selected from the N recognition network models, and M is a positive integer less than N; The M recognition network models are selected as the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions.

13. The method according to any one of claims 1 to 12, wherein adjusting the image resolution of the original part image according to the image resolution adapted to the first recognition network model by the image signal processor to obtain the adjusted part image corresponding to the first recognition network model includes: The image signal processor determines the image size adapted to the first recognition network model based on the image resolution adapted to the first recognition network model. Based on the image size adapted to the first recognition network model, the image size of the original part image is adjusted to obtain the adjusted part image corresponding to the first recognition network model.

14. The method according to any one of claims 1 to 13, wherein the processor resource occupancy conditions include one or more of the following: the processing resources occupied by the identification network model in the part identification process are less than a resource threshold; the identification network model does not belong to the top K identification network models among the N identification network models that occupy the most processing resources; the processing latency of the identification network model in the part identification process is less than a latency threshold; and the identification network model does not belong to the top M identification network models among the N identification network models that have the largest corresponding processing latency; where K and M are both positive integers less than N.

15. An image data processing apparatus, applied to a recognition device, the recognition device including an image signal processor, the apparatus comprising: The acquisition module is used to acquire the original image of the identification area, as well as the recognition performance parameters corresponding to the N recognition network models. N is a positive integer; The determining module is used to determine, based on the recognition performance parameters corresponding to the N recognition network models respectively, the first recognition network model whose recognition performance parameters do not meet the processor resource usage conditions from the N recognition network models; An adjustment module is used to adjust the image resolution of the original part image according to the image resolution adapted by the first recognition network model if the image resolution adapted by the first recognition network model is different from the image resolution of the original part image, so as to obtain the adjusted part image corresponding to the first recognition network model. and The transmission module is used to determine a preprocessed image of the adjusted part image based on the adjusted part image, and transmit the preprocessed image of the adjusted part image to the central processing unit. The central processing unit is used to call the first recognition network model to perform part recognition on the preprocessed image of the adjusted part image to obtain a first object identifier corresponding to the adjusted part image. The first object identifier is used to reflect the object to which the identified part belongs.

16. A computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 14.

18. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 14.

Images