Target identification method, device, equipment and storage medium

Abstract

The application discloses a target identification method, device and equipment and a storage medium, and comprises the following steps: acquiring an image to be identified, and determining a target region of interest of the image to be identified; decoding the image content of the target region of interest by taking a minimum coding unit as a decoding unit to obtain local pixel data; preprocessing the local pixel data to obtain processed local pixel data suitable for a deep learning inference engine, and storing the processed local pixel data in a continuous area of an internal memory; and transmitting a memory pointer corresponding to the continuous area to the deep learning inference engine, wherein the memory pointer is used for the deep learning inference engine to read the processed local pixel data from the continuous area to perform target identification and obtain a target identification result. The application can realize real-time target identification with low delay and low power consumption.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more particularly to a target recognition method, apparatus, device, and storage medium. Background Technology

[0002] With the development of artificial intelligence and edge computing, Android-based smart terminals, such as smart screens, edge computing boxes, industrial inspection equipment, and agricultural drones / robots, are increasingly connecting external USB cameras for real-time image recognition and target detection. To balance bandwidth and image quality, high-resolution or high-frame-rate USB cameras generally use the MJPEG compression format to transmit video frames.

[0003] In the Android system, the core of image recognition lies in feeding the data stream captured by the camera into the deep learning model inference engine with the lowest possible latency and the most complete details. However, the traditional Android image processing pipeline often suffers from severe performance bottlenecks when faced with high-resolution, high-frame-rate data streams due to its lengthy links and excessive redundant computations. Summary of the Invention

[0004] This invention provides a target recognition method, apparatus, device, and storage medium to overcome the shortcomings of existing image recognition methods and achieve low-latency, low-power target image recognition.

[0005] In a first aspect, embodiments of the present invention provide a target recognition method comprising: Acquire the image to be identified and determine the target region of interest in the image; Using the smallest coding unit as the decoding unit, the image content of the target region of interest is decoded to obtain local pixel data; The local pixel data is preprocessed to obtain the processed local pixel data adapted to the deep learning inference engine, and the processed local pixel data is stored in a contiguous area of ​​memory. The memory pointers corresponding to the continuous regions are passed to the deep learning inference engine. The memory pointers are used by the deep learning inference engine to read the processed local pixel data from the continuous regions for target recognition and obtain the target recognition result.

[0006] In a second aspect, embodiments of the present invention provide a target identification device, comprising: The acquisition module is used to acquire the image to be recognized and determine the target region of interest in the image to be recognized; The decoding module is used to decode the image content of the target region of interest using the smallest coding unit as the decoding unit, and obtain local pixel data. The preprocessing module is used to preprocess the local pixel data to obtain the processed local pixel data adapted to the deep learning inference engine, and store the processed local pixel data in a continuous area of ​​memory. The recognition module is used to pass the memory pointer corresponding to the continuous region to the deep learning inference engine. The memory pointer is used by the deep learning inference engine to read the processed local pixel data from the continuous region for target recognition and obtain the target recognition result.

[0007] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a target recognition method.

[0008] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described target recognition method.

[0009] The aforementioned target recognition method, apparatus, device, and storage medium use the smallest coding unit as the decoding unit, and only decode the image content of the target's region of interest, skipping the full decoding operation of non-regions of interest. The decoding computation is proportional to the area of ​​the target's region of interest, avoiding invalid pixel decoding computation. The memory pointer passing mechanism replaces the repeated deep copying of the entire frame data, which can effectively reduce CPU computing power and memory bandwidth usage. Local decoding and memory pointer passing significantly reduce the processing load, enabling low- and mid-range Android hardware platforms to support high-resolution, high-frame-rate real-time recognition, and broadening the feasibility of application scenarios such as industrial inspection, agricultural drones, and edge computing boxes. Attached Figure Description

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] Figure 1 This is a schematic diagram of an application environment for a target recognition method according to an embodiment of the present invention; Figure 2 This is a flowchart of a target recognition method according to an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart of step S102; Figure 4 yes Figure 2 A flowchart of step S103; Figure 5 This is another flowchart of the target recognition method in one embodiment of the present invention; Figure 6 This is a flowchart of a target recognition device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0012] To make the technical problems solved, the technical solutions, and the beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0013] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0014] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0015] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0016] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0017] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0018] Existing technologies for target recognition typically employ a serial, hierarchical, and full-processing model, which includes the following steps: Step 1: Transmission and initial decoding: The USB camera transmits the MJPEG stream to the Linux kernel layer of the Android system via the UVC protocol, and the UVC driver stores it in a hardware buffer or shared memory.

[0019] Step 2: Full Decoding: The Android underlying multimedia framework calls the standard decoder to perform full entropy decoding and inverse discrete cosine transform (IDCT) on the entire MJPEG image frame, generating an uncompressed full-resolution YUV format image.

[0020] Step 3: Cross-layer transmission: The full YUV image data is transmitted through Android's Binder mechanism, MediaCodec framework or JNI interface, passing through the Framework layer by layer, and finally delivered to the application layer.

[0021] Step 4: Memory Copy: To avoid blocking the underlying acquisition queue, the application layer typically needs to perform a deep copy of the entire YUV image frame, copying the data from the system shared buffer to the application layer's own runtime memory space. The memory copy overhead is significant.

[0022] Step 5: Serial preprocessing: The application layer performs serial software operations on the copied full YUV image, including cropping (cutting out sub-images based on ROI), scaling (scaling down to the model input resolution, such as 640×640), color gamut conversion (YUV to RGB), and normalization (subtracting the mean and dividing the variance).

[0023] Step 6: Inference and Post-processing: The generated RGB matrix is ​​fed into the deep learning inference engine to perform forward propagation and output the target recognition result.

[0024] The above-mentioned solutions have the following problems in high-resolution and high-frame-rate scenarios: First, full decoding causes a lot of invalid computation, but existing technologies must decode the entire frame image completely, resulting in a significant waste of decoding computation. Second, multiple rounds of full-frame memory copying lead to a surge in memory bandwidth pressure. Third, the preprocessing link is lengthy, and the higher the frame rate, the more likely the CPU is to be fully loaded and reduce its frequency, resulting in long end-to-end latency, frequent frame dropping, and inability to meet the recognition requirements of high real-time scenarios.

[0025] This invention provides a target recognition method that can be applied to, for example... Figure 1 The application environment is shown. Specifically, this target recognition method is applied in a target recognition system, which includes a client and a server. The client and server communicate via a network to acquire the image to be recognized, local pixel data, and perform target recognition. The client, also known as the user terminal, refers to the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0026] In one embodiment, please refer to Figure 2 A target recognition method is provided, comprising the following steps: Step S101: Acquire the image to be recognized and determine the target region of interest in the image to be recognized; Step S102: Using the smallest coding unit as the decoding unit, decode the image content of the target region of interest to obtain local pixel data; Step S103: Preprocess the local pixel data to obtain processed local pixel data adapted to the deep learning inference engine, and store the processed local pixel data in a continuous area of ​​memory.

[0027] Step S104: Pass the memory pointer corresponding to the continuous region to the deep learning inference engine. The memory pointer is used by the deep learning inference engine to read the processed local pixel data from the continuous region for target recognition and obtain the target recognition result.

[0028] The image to be identified is a raw digital image output by an image acquisition device (such as a USB camera) that has not yet undergone target recognition processing. In this invention, the image to be identified typically originates from a single frame in a real-time video stream from a camera. Its data format can be MJPEG compression or other image compression formats based on block coding units. The Region of Interest (ROI) refers to a local area in the image to be identified that requires target recognition analysis. In target recognition applications, deep learning models typically only need to focus on a specific local area in the image (e.g., the area where the target object appears), rather than the entire content of the image. The ROI is defined using coordinate information (such as the coordinates of the top-left and bottom-right corners or bounding box parameters).

[0029] As an example, in step S101, the original image frame to be identified is first acquired by the image acquisition device. This image frame is stored in the system memory in a compressed format such as MJPEG. Subsequently, the local region in the current image to be identified is determined as the region of interest, and the position information of this region in the image is recorded in coordinate form, such as crop-left, crop-top, crop-right, crop-bottom.

[0030] In image compression formats using block coding (such as JPEG / MJPEG), the Minimum Coded Unit (MCU) is the basic unit of organized encoded data. In the JPEG standard, an MCU typically consists of several 8×8 or 16×16 pixel blocks and is the smallest independently processable unit for entropy decoding and inverse discrete cosine transform (IDCT). A frame of image data is composed of multiple MCUs, and the decoder processes each block independently of the MCU. Local pixel data is pixel-level data obtained by decoding only the set of MCUs corresponding to the region of interest (ROI). Unlike the complete image data obtained after full-frame decoding, local pixel data only contains the decoding results of the MCU blocks covered by the ROI and its boundaries, and does not contain any pixel information from non-ROI regions; therefore, the data volume is much smaller than that of the full-frame decoding result.

[0031] As an example, in step S102, the coordinates of the target region of interest (ROI) are passed to the underlying decoder, which encapsulates local decoding features. Utilizing the physical block characteristics of the smallest coding unit (MCU), Huffman decoding and inverse discrete cosine transform (ICT) are performed on the image content of the ROI, skipping the decoding processing of the MCUs corresponding to non-ROI regions, thus obtaining local pixel data covering only the ROI. Compared to existing technologies that require complete entropy decoding and ICT for the entire frame, this scheme compresses the decoding computation to a level proportional to the area of ​​the ROI, significantly reducing the preprocessing time for a single frame image from acquisition to input into the inference engine.

[0032] Preprocessing refers to a series of data processing operations performed to convert the local pixel data decoded from the compressed bitstream into a standardized data format adapted to the input requirements of the deep learning inference engine. A contiguous region refers to a segment of storage space in the system's physical or virtual memory address space with consecutive address numbers and no gaps, enabling the deep learning inference engine to efficiently and completely read all input data using a starting pointer and a fixed offset.

[0033] As an example, in step S103, the local pixel data obtained in step S102 is preprocessed to convert it into the input data format required by the deep learning inference engine (such as RGB format, specific resolution, normalized numerical range), resulting in processed local pixel data. This processed local pixel data is stored in a contiguous physical address region (contiguous region) in memory, enabling the deep learning inference engine to continuously read complete data blocks by adding an offset to the starting address.

[0034] A memory pointer is a variable that points to the starting address of a specific storage unit in computer memory; its value is the starting address number of that storage unit in the memory address space. A deep learning inference engine is a software runtime environment or low-level computing library specifically designed to run pre-trained deep learning models. It receives input data (such as images, text, and audio), loads pre-trained model parameters, performs forward propagation computation, and outputs inference results (such as target category, coordinate position, etc.).

[0035] As an example, in step S104, the starting address of the continuous region from step S103 is used as a memory pointer and directly passed to the deep learning inference engine. The deep learning inference engine uses this memory pointer to read the processed local pixel data from the continuous region and perform forward propagation calculations. It then parses the data output from the forward propagation calculations to obtain the target recognition result, including the target coordinates and target category. This embodiment eliminates the multi-round full-frame memory copying across levels and modules in traditional schemes by storing the processed pixel data in a continuous region of memory and passing the memory pointer, further reducing data transfer time. Simultaneously, in conjunction with the aforementioned decoding of image content only in the target's region of interest, the end-to-end latency from camera exposure to the start of inference by the deep learning inference engine is significantly reduced, effectively avoiding frame loss caused by slow pre-processing.

[0036] In one embodiment, step S101, namely acquiring the image to be identified, includes: Step S1011: Obtain the video stream output by the camera, extract image frames frame by frame from the video stream, and obtain the image to be recognized.

[0037] In this context, a video stream refers to a data sequence composed of multiple frames of image data transmitted continuously from a camera at a certain frame rate (such as 30 or 60 frames per second) in chronological order. In USB camera scenarios, video streams are typically transmitted in MJPEG compression format. Each frame is an independent JPEG compressed data packet in the bitstream, and frames are distinguished from each other by specific markers (such as the JPEG SOI marker 0xFFD8 and EOI marker 0xFFD9). Video streams are characterized by real-time and continuous data transmission, with data continuously output from the camera until acquisition stops.

[0038] As an example, in step S1011, a camera is connected via a USB interface, and the real-time video stream data output by the camera is received using the UVC (USB Video Class) protocol. The video stream is a sequence of multiple frames of MJPEG compressed image data arranged in chronological order. From this video stream, by detecting the start and end markers of each frame of image data, independent compressed image frames are separated frame by frame, and each frame is used as an image to be identified. The raw MJPEG compressed bitstream resides directly in the underlying hardware buffer or shared memory. The application layer only holds a pointer to the memory address of the raw MJPEG compressed bitstream. That is, the extracted image to be identified is temporarily stored in system memory in MJPEG compression format, and its memory address is recorded for use in subsequent steps. No decoding operation is performed during this process, and the data compression state is maintained.

[0039] This embodiment extracts image frames from the video stream frame by frame, but does not decode the frame data. Instead, it temporarily stores the raw MJPEG compressed data in memory. This allows subsequent steps to directly locate and decode based on the smallest coding unit in the compressed bitstream, avoiding the redundant operation path of full decoding followed by cropping in traditional schemes. This is the prerequisite for achieving local decoding and reducing computational load in this invention. Simultaneously, by processing the video stream frame by frame, each frame immediately enters the subsequent local decoding and inference process after image extraction. Frame-to-frame pipeline parallel processing can be formed, meaning that while inference is being performed on the current frame, extraction and local decoding of the next frame are already underway, thus supporting high frame rate real-time target recognition at 30fps or even 60fps.

[0040] In one embodiment, step S101, determining the target region of interest in the image to be identified includes: Step S1012: Determine the target region of interest in the image to be recognized based on preset coordinates; or, Step S1013: Obtain the historical target recognition result corresponding to the previous frame of the image to be recognized; Step S1014: Determine the target region of interest of the image to be identified based on the historical target recognition results.

[0041] As an example, in step S1012, the target region of interest (ROI) of the image to be recognized is determined according to preset coordinates. Specifically, a fixed rectangular region coordinate parameter is preset, including the coordinates of the upper left corner and the lower right corner, and this rectangular region is directly used as the target ROI of the current image to be recognized. The preset coordinates can be pre-configured according to the business requirements of the application scenario. For example, in an industrial defect detection scenario with a fixed camera position, the target object always appears in a fixed position on the screen, so this position area is directly set as the target ROI.

[0042] As an example, in step S1013, the historical target recognition result corresponding to the previous frame image to be recognized is obtained. The historical target recognition result includes at least the target bounding box coordinates identified in the previous frame; based on the target bounding box coordinates in the historical target recognition result.

[0043] As an example, in step S1014, the target region of interest in the current image is determined by combining the target motion prediction between the current frame and the previous frame (such as based on the displacement estimation of the target between consecutive frames). For example, if the target was identified in the coordinate region in the previous frame, the coordinate region can be appropriately expanded by a margin (such as by 20%) and used as the target region of interest in the current frame to compensate for the slight movement of the target between frames.

[0044] In this embodiment, on the one hand, in applications with fixed camera positions and fixed scenes (such as assembly line quality inspection and fixed-angle facial recognition attendance), the target object always appears in a fixed position in the frame. The target region of interest (ROI) is directly determined by preset coordinates without any additional calculations or data dependencies, and the computational overhead of the ROI determination process is zero. The preset coordinates are not affected by the recognition results of the previous frame, and the ROI in subsequent frames will not shift due to recognition deviations or false detections in a certain frame, exhibiting extremely high reliability in stable scenarios. On the other hand, in scenarios where the target moves in the frame (such as drone tracking and robot vision navigation), the ROI of the current frame is dynamically determined by the recognition results of the previous frame, allowing the ROI to move with the target's movement, ensuring that the target is always within the ROI range and guaranteeing the effectiveness of local decoding. These two methods cover two major categories of practical application needs: fixed and dynamic scenarios. Users can flexibly choose or combine them according to specific business scenarios. For example, a static preset method can be used when the first frame or the target is lost, while a dynamic tracking method can be used during normal tracking, giving this invention broad environmental adaptability and deployment flexibility.

[0045] Please see Figure 3 In one embodiment, step S102, which involves decoding the image content of the target region of interest using the smallest coding unit as the decoding unit, includes: Step S1021: Calculate the set of smallest coding units that intersect with the boundary coordinates based on the boundary coordinates of the target region of interest; Step S1022: Decode the set of smallest coding units.

[0046] The set of minimum coding units (MCUs) refers to the set of all minimum coding units in the MJPEG compressed bitstream of the image to be identified that intersect with the boundary coordinates of the target region of interest (ROI). Since the minimum coding unit is the smallest independent processing unit in image compression coding, each minimum coding unit corresponds to an independent segment of coded data (including Huffman coding, quantization coefficients, etc.) in the compressed bitstream. By calculating the row and column positions of the top-left and bottom-right corner coordinates of the ROI within the entire frame's minimum coding unit grid, it is possible to determine which minimum coding units intersect with the ROI. The set of these minimum coding units constitutes the minimum coding unit set.

[0047] As an example, in step S1021, firstly, the boundary coordinates of the target region of interest determined in step S1012 are obtained. Then, based on the minimum coding unit size of the image to be recognized, the pixel coordinates of the target region of interest are mapped to the minimum coding unit grid coordinates.

[0048] As an example, in step S1022, based on the offset address and data length of each minimum coding unit in the set of minimum coding units calculated in step S1021, only the compressed bitstream segment corresponding to that set is read from the original MJPEG compressed data. Huffman decoding, inverse quantization, and inverse discrete cosine transform (IDCT) are sequentially performed on the read bitstream segment to obtain the pixel block data corresponding to each minimum coding unit. The decoded pixel blocks of each minimum coding unit are then concatenated according to their positions in the image to obtain local pixel data covering the target region of interest. No decoding operation is performed on minimum coding units outside the set of minimum coding units.

[0049] Please see Figure 4 In one embodiment, step S103, the preprocessing of local pixel data to obtain processed local pixel data adapted to the deep learning inference engine, includes: Step S1031: Scale the local pixel data to obtain scaled local pixel data; Step S1032: Perform color gamut conversion on the scaled local pixel data to obtain color gamut converted local pixel data; Step S1033: Perform pixel normalization calculation on the local pixel data after color gamut conversion to obtain the processed local pixel data.

[0050] Scaling refers to scaling the decoded local pixel data from its original resolution to the target input resolution required by the deep learning inference engine.

[0051] As an example, in step S1031, the decoded local pixel data is acquired, with a resolution equal to the original resolution of the smallest coding unit covered by the target region of interest. Based on the target input resolution required by the deep learning inference engine (e.g., 640×640 or 416×416), scaling is performed on the local pixel data. Alternatively, during the inverse discrete cosine transform, the sampling scaling capability of the JPEG decoder can be directly utilized to achieve downsampling close to the target resolution. The scaling process can employ image interpolation algorithms, including but not limited to bilinear interpolation or nearest-neighbor interpolation, to map the local pixel data from the original resolution to the target resolution, obtaining the scaled local pixel data.

[0052] In this invention, color gamut conversion transforms scaled local pixel data from one color space to the color space required by the deep learning inference engine. The data decoded from MJPEG in this invention is typically in YUV format (luminance-chrominance separation), while most deep learning models require RGB format. Therefore, a color space conversion formula is needed to convert YUV pixel values ​​to RGB pixel values.

[0053] As an example, in step S1032, scaled local pixel data is acquired, whose color space is the inherent color format after MJPEG decoding. Based on the color space format required by the deep learning inference engine, such as RGB, a color gamut conversion is performed on the scaled local pixel data. The color gamut conversion is achieved through a color space conversion formula, for example, converting YUV pixel values ​​to RGB pixel values. Specifically, while decoding and scaling the smallest coding unit block, the decoder's output is no longer written to intermediate YUV memory. Instead, register-level concurrent acceleration is achieved using the ARM NEON instruction set, directly executing the YUV-to-RGB color gamut conversion formula on-site, or the decoded local YUV components are directly used as input, and the underlying shader (OpenGLES Shader) performs high-speed color gamut conversion in the GPU cache (VRAM).

[0054] Pixel normalization is a process of mapping pixel values ​​from the original integer range to the numerical range used during model training, typically a floating-point range.

[0055] As an example, in step S1033, after obtaining the local pixel data after color gamut conversion, a normalization calculation is performed on each pixel value according to the pixel value distribution parameters used during the training of the deep learning inference engine. The normalized pixel values ​​are of floating-point type, and all the normalized pixel data are written sequentially into a continuous area of ​​memory to obtain the processed local pixel data.

[0056] In this embodiment, through three sub-steps—scaling, color gamut conversion, and pixel normalization—the decoded local pixel data is gradually converted from its original state of "original resolution, YUV format, integer range" to a standardized data format of "fixed resolution, RGB format, normalized floating-point range." This standardized data format precisely matches the input interface requirements of the deep learning inference engine, enabling the inference engine to directly read and execute forward propagation calculations without requiring any additional data format adaptation operations within the inference engine. This reduces the preprocessing burden on the inference engine and improves overall recognition efficiency.

[0057] Please see Figure 5 In one embodiment, a target recognition method further includes: Step S105: Based on the target recognition result, overlay the target bounding box onto the image to be recognized to obtain the target image; Step S106: Output and display the target image.

[0058] The target bounding box is a rectangular box drawn on the image to identify the location of the target based on its coordinates in the image from the target recognition results. This bounding box is typically drawn with a boundary line (such as a solid or dashed line of a specific color), and the area within the box is the region where the identified target object is located. The coordinate information of the target bounding box comes from the target coordinates output by the deep learning inference engine in step S104. During drawing, category label text and confidence scores can be added according to the target category. The target image is the output image generated by overlaying the target bounding box and related annotation information onto the original image to be recognized. The target image retains all the content of the original image, while visually indicating the location, category, and confidence score of the target object recognized by the model through the bounding box, allowing users or downstream systems to directly view or use it.

[0059] As an example, in step S105, the target recognition result output in step S104 is obtained. The target recognition result includes at least the target's coordinate position in the image and the target category. A rectangular bounding box is drawn on the original image to be recognized based on the target coordinates, and optionally, category label text and confidence scores are added above or near the bounding box. The bounding box is drawn using a preset color and line width, ensuring that the bounding box is clearly visible on the image without obscuring key image content. After drawing, a target image with the target bounding box superimposed is obtained.

[0060] As an example, in step S106, the target image obtained in step S105 is output to a display device for display. The output method includes, but is not limited to: rendering the target image onto a SurfaceView or TextureView in the Android system for real-time display, or encoding the target image into a video stream and transmitting it via a wireless network to a remote monitoring terminal for display. During the display process, as the video stream continues to be input, steps S101 to S105 are repeated to achieve real-time target recognition and annotation results display for each frame of the video stream.

[0061] Please see Figure 6 In one embodiment, a target identification device 300 is provided, which corresponds one-to-one with the target identification method in the above embodiments. The target identification device includes: The acquisition module 301 is used to acquire the image to be recognized and determine the target region of interest in the image to be recognized; Decoding module 302 is used to decode the image content of the target region of interest using the smallest coding unit as the decoding unit to obtain local pixel data; The preprocessing module 303 is used to preprocess the local pixel data to obtain the processed local pixel data adapted to the deep learning inference engine, and store the processed local pixel data in a continuous area of ​​memory. The recognition module 304 is used to pass the memory pointer corresponding to the continuous region to the deep learning inference engine. The memory pointer is used by the deep learning inference engine to read the processed local pixel data from the continuous region for target recognition and obtain the target recognition result.

[0062] Optionally, in one embodiment, the acquisition module 301 is further configured to acquire the video stream output by the camera, extract image frames frame by frame from the video stream, and obtain the image to be identified.

[0063] Optionally, in one embodiment, the acquisition module 301 is further configured to determine the target region of interest of the image to be identified based on preset coordinates.

[0064] Optionally, in one embodiment, the acquisition module 301 is further configured to: Obtain the historical target recognition results corresponding to the previous frame of the image to be recognized; Based on the historical target recognition results, the target region of interest in the image to be recognized is determined.

[0065] Optionally, in one embodiment, the decoding module 302 is further configured to: Calculate the set of minimum coding units within the coordinates of the region of interest. Decoding is performed on the set of minimum coding units.

[0066] Optionally, in one embodiment, the preprocessing module 303 is further configured to: The local pixel data is scaled to obtain scaled local pixel data; Perform color gamut conversion on the scaled local pixel data to obtain color gamut converted local pixel data; Pixel normalization is performed on the local pixel data after color gamut conversion to obtain the processed local pixel data.

[0067] Optionally, in one embodiment, the target recognition device further includes: The generation module 305 is used to overlay target bounding boxes on the image to be recognized based on the target recognition result to obtain the target image; The output display module 306 is used to output and display the target image.

[0068] Specific limitations regarding the target identification device 300 can be found in the above description of the optical network unit fault diagnosis method, and will not be repeated here. Each module in the aforementioned target identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0069] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores images to be identified, local pixel data, etc. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a target recognition method.

[0070] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the target recognition method described in the above embodiment; to avoid repetition, further details are omitted here. The computer-readable storage medium may be non-volatile or volatile.

[0071] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0073] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A target recognition method, characterized in that, include: Acquire the image to be identified and determine the target region of interest in the image to be identified; Using the smallest coding unit as the decoding unit, the image content of the target region of interest is decoded to obtain local pixel data; The local pixel data is preprocessed to obtain processed local pixel data adapted to the deep learning inference engine, and the processed local pixel data is stored in a continuous area of ​​memory. The memory pointer corresponding to the continuous region is passed to the deep learning inference engine. The memory pointer is used by the deep learning inference engine to read the processed local pixel data from the continuous region for target recognition and obtain the target recognition result.

2. The target recognition method as described in claim 1, characterized in that, Obtaining the image to be recognized includes: The video stream output by the camera is acquired, and image frames are extracted frame by frame from the video stream to obtain the image to be identified.

3. The target recognition method as described in claim 1, characterized in that, Determining the target region of interest in the image to be identified includes: The target region of interest in the image to be identified is determined based on preset coordinates.

4. The target recognition method as described in claim 1, characterized in that, Determining the target region of interest in the image to be identified includes: Obtain the historical target recognition results corresponding to the previous frame of the image to be recognized; Based on the historical target recognition results, the target region of interest in the image to be recognized is determined.

5. The target recognition method as described in claim 1, characterized in that, Decoding the image content of the target region of interest using the smallest coding unit as the decoding unit includes: Based on the boundary coordinates of the target region of interest, calculate the set of minimum coding units that intersect with the boundary coordinates; Decoding is performed on the set of minimum coding units.

6. The target recognition method as described in claim 1, characterized in that, The preprocessing of local pixel data to obtain processed local pixel data adapted to the deep learning inference engine includes: The local pixel data is scaled to obtain scaled local pixel data; Perform color gamut conversion on the scaled local pixel data to obtain color gamut converted local pixel data; Pixel normalization is performed on the local pixel data after color gamut conversion to obtain the processed local pixel data.

7. The target recognition method as described in claim 1, characterized in that, The method further includes: Based on the target recognition result, a target bounding box is superimposed on the image to be recognized to obtain the target image; Output and display the target image.

8. A target recognition device, characterized in that, include: An acquisition module is used to acquire an image to be identified and determine the target region of interest of the image to be identified; The decoding module is used to decode the image content of the target region of interest using the smallest coding unit as the decoding unit, and obtain local pixel data. The preprocessing module is used to preprocess the local pixel data to obtain processed local pixel data adapted to the deep learning inference engine, and store the processed local pixel data in a continuous area of ​​memory. The recognition module is used to pass the memory pointer corresponding to the continuous region to the deep learning inference engine. The memory pointer is used by the deep learning inference engine to read the processed local pixel data from the continuous region for target recognition and obtain the target recognition result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the target recognition method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the target recognition method as described in any one of claims 1-7.

Images