Method, system, and storage medium for object recognition

Abstract

A method, system and storage medium for object recognition are provided, wherein the object recognition system comprises a GPU server and a general server, and the method comprises: the GPU server acquires a video stream, parses the video stream into a plurality of image frames; infers objects in the plurality of image frames to obtain inference results, the inference results comprising object types of the objects in the image frames; adds an identification of the object type in the image frames according to the inference results of the objects; packs the image frames with the added identification according to TCP, and sends data of the packed image frames to the general server; and the general server transcodes the received data of the packed image frames to obtain a target video stream. By separating video inference and video transcoding to be executed by different servers, the concurrency of the server used for inference is improved, which is conducive to realizing real-time detection of objects, and by separating video transcoding to be executed by the general server, the occupation of GPU resources is reduced, and the cost of the object recognition system is reduced.

Description

Technical Field

[0001] This invention relates to the field of video image processing, and more specifically, to a method, system, and storage medium for object recognition. Background Technology

[0002] Currently, the detection of ships on water mainly relies on images acquired from satellites. However, acquiring these images usually takes a long time and cannot be done in real time. Therefore, how to achieve real-time detection of ships on water is an urgent problem to be solved. Summary of the Invention

[0003] This application provides a method, system, and storage medium for object recognition, enabling real-time detection of waterborne vessel hulls at a low cost.

[0004] In a first aspect, a method for object recognition is provided. The object recognition system includes a GPU server and a general-purpose server. The method includes:

[0005] The GPU server acquires the video stream captured by the video acquisition device and parses the video stream into multiple image frames. It then uses an object recognition model to infer the objects in the multiple image frames, obtaining inference results for the objects in the multiple image frames. The object recognition module is obtained using standard model transfer learning, and the inference results include the object types of the objects in the image frames. Based on the object inference results, an identifier for the object type is added to the image frames. The data corresponding to the image frames with added identifiers is packetized according to the Transmission Control Protocol (TCP), and the packetized data is sent to the general-purpose server.

[0006] The general server decodes the received packet data into image frames corresponding to the data, and compresses the image frames to convert them into a target video stream.

[0007] Secondly, an object recognition system is provided, including: a GPU server and a general-purpose server;

[0008] The GPU server includes: an acquisition module, a video inference module, a video visualization module, and a video distribution module;

[0009] The acquisition module is used to acquire the video stream captured by the video acquisition device and parse the video stream into multiple image frames;

[0010] The video inference module is used to infer the objects in the multiple image frames using an object recognition model to obtain the inference results of the objects in the multiple image frames. The object recognition module is obtained using standard model transfer learning, and the inference results include the object type of the objects in the image frames.

[0011] The video visualization module is used to add an identifier of the object type to the image frame based on the reasoning result of the object;

[0012] The video distribution module is used to encapsulate the data corresponding to the image frame with added identifier according to the transmission control protocol TCP, and send the encapsulated data to the general server;

[0013] The general-purpose server includes a data decoding module and an image compression module;

[0014] The data decoding module is used to decode the received packet data into image frames corresponding to the data.

[0015] The image compression module compresses the image frames and converts them into a target video stream.

[0016] Thirdly, an object recognition system is provided, including:

[0017] Processor; and

[0018] Memory for storing the executable instructions of the processor;

[0019] The processor is configured to perform the object recognition method described above by executing executable instructions.

[0020] Fourthly, a storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps of the above-described object recognition method.

[0021] General-purpose servers can be ordinary cloud servers, while GPU servers can be dedicated cloud servers for graphics processing. GPU servers are more expensive, while general-purpose servers are less expensive. By performing video transcoding operations, which consume a lot of CPU resources, on ordinary cloud servers, the computing power requirements of GPU servers used for video inference can be reduced, thereby reducing the cost of object recognition systems.

[0022] Traditional object recognition methods can only process one 4K video stream at a time for a GPU server with the same computing power. However, the distributed object recognition method of this application can infer at least four 4K video streams, which increases the concurrency of video inference and is beneficial for real-time object detection. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the detection results of target detection in related technologies.

[0024] Figure 2 This is a schematic diagram of an application scenario of the object recognition method according to the authorized embodiments.

[0025] Figure 3 This is a schematic flowchart of an object recognition method provided in an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the functional modules of a GPU server in an object recognition system according to an embodiment of this application.

[0027] Figure 5 This is a schematic diagram of the functional modules of a general server in an object recognition system according to an embodiment of this application.

[0028] Figure 6 This is a schematic flowchart illustrating the video reasoning process in an embodiment of this application.

[0029] Figure 7 This is a schematic flowchart illustrating the video visualization process in an embodiment of this application.

[0030] Figure 8 This is a schematic illustration of a logo background image with a blue transparent background.

[0031] Figure 9 This is a schematic illustration of a logo background image using a dark green transparent background.

[0032] Figure 10 This is a schematic flowchart of a video distribution process according to an embodiment of this application.

[0033] Figure 11 This is a schematic flowchart of the video transcoding process according to an embodiment of this application.

[0034] Figure 12 This is a schematic block diagram of an object recognition system provided according to an embodiment of this application.

[0035] Figure 13 This is a schematic block diagram of another object recognition system provided according to an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art without creative effort regarding the embodiments of this application are within the scope of protection of this application.

[0037] It should be understood that the accompanying drawings are schematic illustrations of this application, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. In the following description, numerous specific details are provided to give a full understanding of exemplary embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced with one or more specific details omitted, or other methods, components, steps, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0038] It should also be understood that some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different networks, processor devices, or microcontroller devices.

[0039] Cloud technology refers to a managed technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0040] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0041] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0042] There are many types of clouds, such as private cloud, public cloud, and hybrid cloud.

[0043] A private cloud is a cloud infrastructure and hardware / software resources created within a firewall, allowing various departments within an organization or enterprise to share resources within a data center. In addition to hardware resources, creating a private cloud typically involves cloud equipment (IaaS, Infrastructure as a Service) software.

[0044] Private cloud computing comprises three layers: cloud hardware, cloud platform, and cloud services. The key difference is that cloud hardware consists of the user's own personal computer or server, rather than the cloud computing provider's data center. Cloud computing providers build data centers to provide public cloud services to millions of users, thus requiring hundreds of thousands or even millions of servers. Private cloud computing, for individuals, serves only family and friends; for businesses, it serves only their own employees, customers, and suppliers. Therefore, an individual's or business's own personal computer or server is sufficient to provide cloud services.

[0045] Public cloud typically refers to a cloud service provided by a third-party provider to users. Public clouds are generally accessible via the Internet and may be free or inexpensive. The core attribute of a public cloud is shared resource service. Many instances of this type of cloud exist, providing services across today's open public networks.

[0046] Hybrid cloud, which combines public cloud and private cloud, has become a major model and development direction of cloud computing in recent years. Private cloud primarily targets enterprise users; for security reasons, enterprises prefer to store data in private clouds, but at the same time, they also want access to the computing resources of public clouds. Hybrid cloud is increasingly adopted in this context, combining and matching public and private clouds to achieve optimal results. This personalized solution achieves both cost-effectiveness and security.

[0047] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI software technology mainly includes computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0048] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0049] Object detection and recognition refers to finding objects in a scene (image), including two processes: detection (where) and recognition (what). In related technologies, object detection suffers from poor visualization, such as... Figure 1 As shown, the main process involves drawing some boxes on the detected target object and adding the object's logo.

[0050] The detection of vessels on water is a type of object detection. The recognition and display of vessels on water typically involves multiple cloud services, such as live streaming services, transcoding services, model training services, and model inference services. Implementing this using public clouds presents challenges, such as the need for in-depth development and integration of different SDKs for each cloud service, resulting in high R&D costs. Furthermore, public clouds cannot be used for private scenarios, and each of the aforementioned cloud services needs to be purchased separately, which is also costly. Therefore, achieving real-time detection of vessels on water requires even more server resources. Thus, finding a low-cost solution for real-time detection of vessels on water is an urgent problem to be solved.

[0051] Figure 2 This is a schematic diagram illustrating an application scenario of an object recognition system according to an embodiment of this application.

[0052] It should be understood that the object recognition system of this application embodiment can be applied to the recognition of water conservancy vessel hulls, real-time monitoring of vessel hulls, structuring of vessel hull videos, recognition or monitoring of road vehicles, etc. For example, the object recognition system can be a vessel recognition system or a vessel monitoring system, used to realize real-time monitoring of water conservancy, reduce the cost of manual monitoring, and can also provide core input for real-time monitoring alarms, and perform real-time vessel early warning, etc.

[0053] like Figure 2As shown, the object recognition system 13 may include a GPU server 131 and a general-purpose processor 132, which can share different video processing tasks in object recognition. For example, video inference and video transcoding, which consume more CPU resources, can be performed by the GPU server 131 and the general-purpose processor 132 respectively. In this case, the GPU server 131 can be called the inference server, and the general-purpose processor 132 can be called the transcoding server. By offloading video inference and video transcoding to different servers, the concurrency of the inference server is increased. That is, for the same computing power, the inference server can perform inference processing on multiple video data in parallel, enabling real-time object detection.

[0054] In some embodiments of this application, the general-purpose server can be a regular cloud server, while the GPU server can be a dedicated graphics processing cloud server. GPU servers are more expensive than general-purpose servers. Performing the CPU-intensive video transcoding operation on a regular cloud server reduces the computational demands on the GPU server and lowers the cost of the object recognition system. For object recognition methods that do not employ a distributed approach, a GPU server with the same computing power can only process one 4K video at a time. However, the distributed approach used in this application allows for inference of at least four video streams, increasing the concurrency of video inference and facilitating real-time object detection.

[0055] Combination Figure 2 This application describes the process of object recognition and display according to one embodiment.

[0056] like Figure 2 As shown, firstly, the video acquisition device 11 (e.g., a camera) acquires video data from the scene to be detected (e.g., a specific sea area or a specific body of water). The video data acquired by the video acquisition device 11 can be streamed to the first streaming media server 12 (or streaming server) via the Real Time Messaging Protocol (RTMP).

[0057] Furthermore, the object recognition system 13 can obtain the video data to be recognized from the first streaming media server 12 and process the video data.

[0058] The object recognition system 13's processing of video data may include the following: video reasoning, image visualization, video distribution, and video transcoding. Video reasoning may include detecting target objects in the video data and determining the type of the target objects. Image visualization may include adding type-corresponding identifiers (logos) to the detected target objects. Video transcoding may include transcoding the video with added logos, for example, transcoding it to a video format playable by the client 15, such as FLV format.

[0059] Specifically, the object recognition system 13 can push the video stream obtained by processing the acquired video data through video inference, video transcoding, etc., to the second streaming media server 14 (or live streaming server). Furthermore, users can play the video stream on the client 15 corresponding to the second streaming media server 14. For example, the client can obtain the video stream based on a preset browser access address, and thus play the video stream.

[0060] It should be understood that the object recognition method of this application embodiment can be applied to systems such as the recognition of water conservancy vessel hulls, real-time monitoring of vessel hulls, structuring of vessel hull videos, and recognition or monitoring of road vehicles. For example, it can be used as a vessel hull monitoring system to complete real-time water conservancy monitoring, reduce the cost of manual monitoring, and provide core input for real-time monitoring alarms, and perform real-time vessel hull early warning, etc.

[0061] The following embodiments use the identification of a ship's hull on water as an example for illustration, but this application is not limited to this.

[0062] Figure 3 A schematic flowchart of an object recognition method 20 according to an embodiment of this application is shown. This method 20 can be applied to... Figure 2 Object recognition system in the middle.

[0063] The object recognition system includes a GPU server and a general-purpose server. The GPU server and the general-purpose server are used to perform different operations in object recognition. In the following embodiments, the operations performed by the GPU server and the general-purpose processor will be described from their respective perspectives.

[0064] For ease of understanding and description, such as Figure 4 As shown, based on the operations performed by the GPU server, the GPU server can be divided into the following functional modules: acquisition module, video inference module, video visualization module, and video distribution module.

[0065] For ease of understanding and description, such as Figure 5 As shown, based on the video processing operations performed by the general-purpose processor, the general-purpose processor can be divided into a data decoding module and an image compression module.

[0066] It should be understood that the GPU server and general-purpose processor in the embodiments of this application may refer to a single server or a server cluster. For example, a general-purpose server can be implemented through a server cluster. This application does not limit this.

[0067] See Figure 3 The method 20 may include at least some of the following:

[0068] S21: The GPU server acquires the video stream captured by the video acquisition device and parses the video stream into multiple image frames. This S21 can be executed by the acquisition module of the GPU server.

[0069] Optionally, the video capture device can be, for example, a... Figure 2 The camera in the image can be any other device with a camera, such as a drone, but this application is not limited to that.

[0070] After capturing the video stream, the video capture device can further push the video stream to a streaming media server (e.g., Figure 2 The first streaming media server 12), for example, a video capture device can push the captured video stream to the streaming media server via the RTMP protocol.

[0071] In some embodiments, the GPU server can pull video streams from the streaming media server via OpenCV, and the video streams can be transmitted to the GPU server via the RTMP protocol.

[0072] Furthermore, the GPU server can perform pre-inference processing on the video stream, such as parsing the video stream into multiple image frames.

[0073] S22 involves using an object recognition model to infer the objects in multiple image frames, obtaining the inference results for the objects in the multiple image frames. This S22 step can be executed by the video inference module within the GPU processor.

[0074] In some embodiments, the object recognition model can be obtained based on transfer learning of a standard model, such as the YOLO model or the SSD (Single Shot Detection) model.

[0075] The Yolo model is a deep learning neural network that solves object detection as a regression problem. It uses a single end-to-end network to process the input of the original image and output the object's location and category.

[0076] SSD: A deep learning neural network. SSD is a popular and powerful object detection network. The network structure includes a base network, auxiliary convolutions, and predictive convolutions.

[0077] In some embodiments, a ship sample library is used as a training set, the ship hulls in the ship sample library are labeled, and the labeled training set is further input into a pre-trained model, and the ship hull recognition model is trained by transfer learning.

[0078] Higher image frame resolution places higher demands on the computing power of GPU servers and increases the complexity of inference. In practical applications, the speed of a ship's movement is usually slow, meaning that the positional difference between the ship and adjacent image frames is not significant. Therefore, the inference frequency can be configured according to the resolution of the image frame to be inferred, i.e., inference is performed once every X image frames. Further inference is then performed based on this inference frequency, which helps to meet the real-time requirements of video inference.

[0079] X is positively correlated with the resolution of the image frame; that is, the larger the resolution of the image frame, the larger X is, and the smaller the resolution of the image frame, the smaller X is.

[0080] As an example, if the resolution of the image frame is 1080P, X can be configured to be 2, that is, inference once every two frames. If the resolution of the image frame is 4K, X can be configured to be 5, that is, inference once every 5 frames.

[0081] In some embodiments, the reasoning result of an object includes the object type. For example, if the object is a ship hull, the object type could be a fishing boat, a sand carrier, a warship, etc.

[0082] In some embodiments, the reasoning result of an object may also include the object's position coordinates, such as the coordinates of the top left corner and the bottom right corner, or the coordinates of the bottom left corner and the top right corner, etc.

[0083] In some embodiments, the reasoning result of an object may also include a confidence level that the object is the inferred object type. For example, the confidence level can be between 0 and 1. The closer the value is to 0, the lower the probability that the object is the inferred object type; the closer the value is to 1, the higher the probability that the object is the inferred object type.

[0084] In some embodiments, before visualizing the image frame, inference results with a confidence level lower than a first threshold can be filtered out. That is, inference results with a confidence level lower than the first threshold are not used as object recognition results, which can improve the accuracy of object recognition.

[0085] As an example rather than a limitation, the first threshold can be 0.25, 0.3, etc.

[0086] Combination Figure 6 Taking the identification of ships on water as an example, the specific implementation process of S21 and S22 will be explained.

[0087] S221, pull the video stream to be inferred and convert the video stream into image frames.

[0088] For example, a video stream can be retrieved from a streaming media server using OpenCV and then further parsed into multiple image frames.

[0089] S222, Load the trained ship hull recognition model.

[0090] The hull identification model can be obtained by transfer learning using a standard model.

[0091] S223, configure the number of image frames used for inference, i.e., the inference frequency, or how many image frames are used to perform inference once.

[0092] Specifically, the number of image frames used for inference can be determined by analyzing the video characteristics of the hydraulic vessel, such as image resolution and the vessel's motion characteristics.

[0093] In some embodiments, the number of image frames X used for inference is positively correlated with the resolution of the image frames; that is, the larger the resolution of the image frames, the larger X is, and the smaller the resolution of the image frames, the smaller X is. Configuring the appropriate inference frequency according to the resolution of the image frames is beneficial for meeting the real-time requirements of inference.

[0094] For example, if the resolution of an image frame is 1080P, inference every two image frames can meet the real-time requirements; if the resolution of an image frame is 4K, inference every five image frames can meet the real-time requirements.

[0095] S224: Using the hull recognition model, inference is performed on the image frames based on the configured number of image frames to obtain the inference result of the hull.

[0096] In some embodiments, the inference results for the hull include the hull type, the hull's position coordinates, and the confidence level.

[0097] S225, filter out inference results with confidence levels below the first threshold.

[0098] The lower the confidence level, the lower the probability that the hull is the inferred hull type. Filtering out these inference results can improve the accuracy of hull identification.

[0099] See also Figure 3After obtaining the reasoning result, in step S23, a hull type identifier (logo) is added to the image frame based on the hull reasoning result.

[0100] In some embodiments of this application, such as Figure 7 As shown, S23 may include:

[0101] S231, Draw the object outline based on the object's position coordinates.

[0102] For example, the reasoning result of an object includes the coordinates of the top left and bottom right corners of the object frame. Therefore, the object frame can be drawn based on the coordinates of the bottom left and bottom right corners of the object frame.

[0103] In some embodiments, all the lines that need to be drawn can be calculated based on the position coordinates of all the objects in the inference, and then drawn all at once, which can reduce the time consumption of object recognition.

[0104] S232, Determine the target position of the object type identifier based on the position of the object frame and the size of the object type identifier.

[0105] In practical applications, since objects are in motion, their positions in an image frame are uncertain. Therefore, setting the object and its corresponding logo to a fixed relative position may not conform to the user's viewing habits in some scenarios. Thus, in this embodiment, the position of the logo can be dynamically adjusted according to the position of the object frame and the size of the logo of the object type.

[0106] In some embodiments of this application, S232 may include:

[0107] Based on the position of the object frame, determine the pre-selected position of the object type label;

[0108] Determine the target location of the object type identifier based on the pre-selected location and size of the object type identifier.

[0109] For example, if the object frame is located in the middle area of ​​the image frame, the pre-selected position of the logo can be the left or right position of the corresponding object, or more specifically, the upper left or upper right position of the object.

[0110] For example, if the object frame is located in the left area of ​​the image frame, the pre-selected position of the logo can be the right side of the corresponding object, or more specifically, the upper right position of the object.

[0111] For example, if the object frame is located in the right region of the image frame, the pre-selected position of the logo can be the left side of the corresponding object, or more specifically, the upper left position of the object.

[0112] In some embodiments of this application, determining the target location of the object type identifier based on the pre-selected location of the object type identifier and the size of the object type identifier may include:

[0113] If the pre-selected position of the logo is the left side of the corresponding object, and the size of the logo extends beyond the left side of the image frame, then the right side of the object is determined as the target position of the logo; or

[0114] If the pre-selected position of the logo is the right side of the corresponding object, and the size of the logo extends beyond the right side of the image frame, then the left side of the object is determined as the target position of the logo; or

[0115] If the pre-selected position of the logo is the left side of the corresponding object, and the size of the logo does not exceed the left side of the image frame, then the left side of the object is determined as the target position of the logo; or

[0116] If the pre-selected position of the logo is the right side of the corresponding object, and the size of the logo does not exceed the right side of the image frame, then the right side of the object is determined as the target position of the logo.

[0117] Specifically, taking the pre-selected position of the logo as the upper left of the corresponding object as an example, in some scenarios, if the object enters the image frame from the left, drawing the logo at the upper left of the corresponding object would cause the logo to extend beyond the left side of the image frame, affecting the display effect. In this case, the right side of the object, such as the upper right, can be used as the target position for the logo. In other scenarios, if the object exits the image frame from the right, drawing the logo at the upper right of the object would cause the logo to extend beyond the right side of the image frame, affecting the display effect. In this case, the left side of the object, such as the upper left, can be used as the target position for the logo.

[0118] In summary, in the embodiments of this application, the relative positions of the logo and the corresponding object in the image frame can be flexibly adjusted according to the position of the object in the image frame and the size of the logo. For example, for an object located in the left edge region of the image frame, the logo of the object can be drawn on the right side of the object, such as the upper right position; for an object located in the right edge region of the image frame, the logo of the object can be drawn on the left side of the object, such as the upper left position; for an object located in the middle region of the image frame, the logo of the object can be drawn on the left side of the object, such as the upper left position, or it can also be drawn on the right side of the object, such as the upper right position.

[0119] Drawing an object-type logo directly on an image frame will cover part of the image, resulting in poor visualization. To improve the visualization effect, in this embodiment, the image of the target location of the logo and the logo itself can be merged, and the merged image can then be drawn on the image frame, thereby improving the visualization effect. (Continue to see...) Figure 7 Specifically, it may include the following steps:

[0120] S233, the image corresponding to the target position of the cropping mark in the image frame is used as the background image.

[0121] For example, if the target position of the logo is the upper left position of the corresponding object, the image of the upper left position of the object can be cropped and used as the background image.

[0122] For example, if the target position of the logo is the upper right position of the corresponding object, the image of the upper right position of the object can be cropped and used as the background image.

[0123] S234, merge the background image and the logo to obtain the target logo image.

[0124] Image fusion is the process of combining two or more images into a new image using specific algorithms. The fusion result can utilize the spatiotemporal correlation and information complementarity of the two (or more) images, and make the fused image provide a more comprehensive and clear description of the scene, thereby enhancing the visualization effect of the image.

[0125] S235, Draw the target logo image onto the target position of the logo.

[0126] By using the image of the target location of the logo as the background image, merging the background image and the logo, and then drawing the merged image at the target location of the logo, the visualization effect of the image can be improved.

[0127] In some embodiments of this application, S235 may include:

[0128] If the pre-selected position of the logo is the left side of the corresponding object, and the size of the logo extends beyond the left side of the image frame (i.e., the target position of the logo is the right side of the corresponding object), then the target logo image will be drawn to the right side of the object; or

[0129] If the pre-selected position of the logo is the right side of the corresponding object, and the size of the logo extends beyond the right side of the image frame (i.e., the target position of the logo is the left side of the corresponding object), then the target logo image will be drawn to the left side of the object; or

[0130] If the pre-selected position of the logo is the left side of the corresponding object, and the size of the logo does not exceed the left side of the image frame, that is, the target position of the logo is the left side of the corresponding object, then draw the target logo image to the left side of the object; or

[0131] If the pre-selected position of the logo is the right side of the corresponding object, and the size of the logo does not exceed the right side of the image frame, that is, the target position of the logo is the right side of the corresponding object, the target logo image is drawn to the right side of the object.

[0132] In some embodiments, S23 can be implemented using the following code:

[0133] def draw_rect_polylines(image, left_top,right_bottom,label, color,drawright, dropsome):

[0134] if label in MAP_LABEL_LOGO:

[0135] (left, top) = left_top

[0136] (right, bottom) = right_bottom

[0137] line_len = 5

[0138] width = abs(left - right)

[0139] height = abs(top - bottom)

[0140] pt1s = np.array([[left + line_len,top],[left ,top],[left,top + line_len]])

[0141] pt2s = np.array([[left + line_len, bottom], [left, bottom], [left,bottom - line_len]])

[0142] pt3s = np.array([[right - line_len, top], [right, top], [right, top +line_len]])

[0143] pt4s = np.array([[right - line_len, bottom], [right, bottom], [right,bottom - line_len]])

[0144] (dy, dx) = left_top

[0145] logo = MAP_LABEL_LOGO[label]

[0146] (logol,logor) = logo

[0147] logo_h, logo_w = logol.shape[0], logol.shape[1]

[0148] xpos = int(dy + width / 2 - logo_w + 10)

[0149] off_y = logo_h - 4 - int(height / 4)

[0150] # Draw on the right

[0151] if drawright:

[0152] xpos = int(dy + width / 2 - 10)

[0153] # Overlay the image

[0154] overlay_transparent(image, logor, xpos, dx - off_y)

[0155] cv2.polylines(image, [pt1s, pt2s, pt3s, pt4s], 0, color, lineType=cv2.LINE_AA, thickness=1)

[0156] # Discard if out of bounds

[0157] elif dropsome:

[0158] if xpos>-50:

[0159] overlay_transparent(image, logol, xpos, dx - off_y)

[0160] cv2.polylines(image, [pt1s, pt2s, pt3s, pt4s], 0, color, lineType=cv2.LINE_AA, thickness=1)

[0161] else:

[0162] overlay_transparent(image, logol,xpos, dx-off_y)

[0163] cv2.polylines(image, [pt1s,pt2s,pt3s,pt4s], 0, color,lineType=cv2.LINE_AA,thickness = 1)

[0164] As described in the foregoing embodiments, in some embodiments of this application, the background image of the target location of the logo can be merged to obtain the target logo image. In other embodiments, the logo can also be merged with other images as the target logo image. For example, the other image can be selected by the user from multiple preset images. Therefore, embodiments of this application can customize the background image of the logo according to user needs. For example, the candidate image for the background image of the logo can be a transparent image of a specific color. The user can select a transparent image of a specific color as the background image of the logo according to the system background, thereby improving the visual effect of the logo display. Figure 8 and Figure 9 The images show the effect of using a blue transparent background and a dark green transparent background as the logo's background image, respectively. S24, the data corresponding to the image frame with the added identifier is packetized according to the Transmission Control Protocol (TCP).

[0165] S25 sends the packet data to the general server.

[0166] S26, the general server decodes the received packet data into image frames corresponding to the data, and compresses the image frames to convert them into a target video stream.

[0167] Steps S24 and S25 can be executed by the video distribution module of the GPU server. Step S26 can be executed by the data decoding module and image compression module of the general-purpose processor. After obtaining the image frame with the logo added based on the above steps S21 to S23, the image frame can be further distributed to the general-purpose processor, which will then perform subsequent video processing, such as video transcoding.

[0168] In some embodiments of this application, data can be transmitted between the GPU server and the general-purpose server via the TCP protocol.

[0169] Specifically, data transmission between the GPU server and the general-purpose server can be implemented based on sockets. The GPU server acts as the socket server, and the general-purpose processor acts as the socket client.

[0170] Combination Figure 10 Explain the specific implementation process of S24 and S25 above. For example... Figure 10 As shown, it may include the following steps:

[0171] S241, the GPU server starts a socket as a server to detect and wait to establish a connection with the client.

[0172] S242, add the data packet of the image frame containing the logo of the merged image.

[0173] Specifically, after receiving a connection request from the client, the GPU server encapsulates the data corresponding to the image frame with the logo of the merged image added, for example, according to the TCP protocol format.

[0174] In some embodiments, the GPU server can adjust the size of the socket buffer according to the resolution of the image frame to improve the processing performance of the GPU server.

[0175] For image frames with a resolution of 4K or 1080P, each image frame occupies a relatively large amount of space. For example, an image frame with a resolution of 1920*1080 occupies 1920*1080*3=6220800 bytes. For a video stream with a transmission rate of 25 frames per second (FPS), it occupies approximately 140Mb of bandwidth per second. Therefore, when transmitting data, it is necessary to increase the size of the socket buffer to improve the processing performance of the GPU server.

[0176] S243 sends the packet data to the client.

[0177] Specifically, the data in the packet is sent to the client according to the TCP protocol.

[0178] like Figure 11 As shown, S26 may include the following steps:

[0179] S261, the general-purpose server, acting as a socket client, initiates a connection request to the socket server (i.e., the GPU server) and establishes a connection with the server.

[0180] S262 receives data packets sent by the GPU server as a socket server and decodes the data packets into image frames.

[0181] S263 performs compression processing on the decoded image frames.

[0182] Specifically, image frames can be compressed into H264 format, for example, by using FFMPEG to compress image frames into H264 format, and then further converted into a video stream of the target format, such as FLV format.

[0183] FFMPEG is an open-source computer program that can be used to record, convert, and stream digital audio and video.

[0184] Furthermore, the video stream in the target format is pushed to a streaming media server, for example, Figure 2 The second streaming media server 14 is used by the client (e.g., to play the video stream) Figure 2 Client 15) retrieves the video stream from the streaming media server and then plays the video stream.

[0185] Since image compression operations consume a lot of CPU resources, in some embodiments, the number of threads corresponding to the resolution of the image frame can be configured, and multi-threading can be used to compress the video, which can improve the efficiency of image compression and reduce the time consumption of object recognition.

[0186] In some embodiments of this application, the general-purpose server can be a regular cloud server, while the GPU server can be a dedicated graphics processing cloud server. GPU servers are more expensive than general-purpose servers. Performing the CPU-intensive video transcoding operation on a regular cloud server reduces the computational demands on the GPU server and lowers the cost of the object recognition system. For object recognition methods that do not employ a distributed approach, a GPU server with the same computing power can only process one 4K video at a time. However, the distributed approach used in this application allows for inference of at least four video streams, increasing the concurrency of video inference and facilitating real-time object detection.

[0187] In some embodiments, S26 can be implemented using the following code:

[0188] def init():

[0189] self.command = ['ffmpeg',

[0190] '-y',

[0191] '-f', 'rawvideo',

[0192] '-vcodec', 'rawvideo',

[0193] '-pix_fmt', 'bgr24',

[0194] '-s', "{}x{}".format(self.width_raw, self.height_raw),

[0195] '-r', str(args.fps),

[0196] '-i', '-',

[0197] '-c:v', 'libx264',

[0198] # Transcoding thread count

[0199] '-threads', '8',

[0200] # Target bitrate

[0201] '-b:v', str(args.bv),

[0202] '-maxrate', str(args.maxrate),

[0203] '-bufsize', str(args.buffersize),

[0204] '-pix_fmt', 'yuv420p',

[0205] '-preset', 'ultrafast',

[0206] '-f', 'flv',

[0207] self.rtmpUrl]

[0208] if self.isClient is False:

[0209] self.openPipe()

[0210] ]>def openPipe(self):

[0211] print(" to open pipe ")

[0212] if self.pipe:

[0213] try:

[0214] self.pipe.stdin.close()

[0215] except Exception:

[0216] print(traceback.format_exc())

[0217] try:

[0218] self.pipe.stdout.close()

[0219] except Exception:

[0220] print(traceback.format_exc())

[0221] try:

[0222] self.pipe.stderr.close()

[0223] except Exception:

[0224] print(traceback.format_exc())

[0225] if len(self.command)>0:

[0226] try:

[0227] # Pipeline configuration

[0228] self.pipe = sp.Popen(self.command, stdin=sp.PIPE)

[0229] except Exception:

[0230] print(traceback.format_exc())

[0231] self.pipe = None

[0232] In summary, this application's embodiments reduce the cost of video transcoding servers by offloading video inference and video compression operations. Video transcoding services can be completed by a general-purpose server, increasing the concurrency of the inference server and supporting multiple video streams. Testing showed that a typical Tesla T4 with 16GB of video memory could only infer one 4K video without a distributed solution, but with the distributed solution of this application, at least four video streams could be inferred. Furthermore, by analyzing the video characteristics of objects and setting the number of image frames used for inference, real-time object recognition in high-definition video can be achieved. Moreover, by using image fusion for logo rendering, the visualization effect is improved. This logo can be customized according to user needs, providing support for adaptation to customer systems and enhancing the user experience.

[0233] The above text combined Figures 3 to 11 The method embodiments of this application are described in detail below, in conjunction with... Figures 12 to 13 The present application describes the device embodiments in detail. It should be understood that the device embodiments correspond to the method embodiments, and similar descriptions can be referred to the method embodiments.

[0234] Figure 12 This is a schematic structural diagram of an object recognition system 1100 according to an embodiment of this application, such as... Figure 12 As shown, the object recognition system 1100 may include: a GPU server 1110 and a general server 1120; wherein, the GPU server 1110 includes: an acquisition module 1111, a video inference module 1112, a video visualization module 1113 and a video distribution module 1114.

[0235] The acquisition module 1111 is used to acquire the video stream captured by the video acquisition device and parse the video stream into multiple image frames.

[0236] The video inference module 1112 is used to infer the objects in the multiple image frames using an object recognition model to obtain the inference results of the objects in the multiple image frames. The object recognition module is obtained by transfer learning using a standard model, and the inference results include the object type of the objects in the image frames.

[0237] The video visualization module 1113 is used to add an identifier of the object type to the image frame based on the reasoning result of the object;

[0238] The video distribution module 1114 is used to encapsulate the data corresponding to the image frame with added identifier according to the Transmission Control Protocol TCP, and send the encapsulated data to the general server;

[0239] The general-purpose server 1120 includes a data decoding module 1121 and an image compression module 1122;

[0240] The data decoding module 1121 is used to decode the received packet data into an image frame corresponding to the data.

[0241] The image compression module 1122 compresses the image frame and converts it into a target video stream.

[0242] In some embodiments of this application, the reasoning result also includes the position coordinates of the object, and the video visualization module 1113 is specifically used for:

[0243] Draw the object outline based on the object's position coordinates;

[0244] Determine the target location of the object type's identifier based on the position of the object frame and the size of the identifier for that object type;

[0245] The image corresponding to the target position of the identifier in the image frame is cropped and used as the background image;

[0246] The background image and the logo are merged to obtain the target logo image;

[0247] Draw the target identification image onto the target location of the identification.

[0248] In some embodiments of this application, the video visualization module 1113 is further used for:

[0249] Based on the position of the object frame, determine the pre-selected position of the object type's identifier;

[0250] Based on the pre-selected location of the object type's identifier and the size of the object type's identifier, determine the target location of the object type's identifier.

[0251] In some embodiments of this application, the video visualization module 1113 is further used for:

[0252] If the pre-selected position of the marker is the left side of the corresponding object, and the size of the marker extends beyond the left side of the image frame, then the right side of the object is determined as the target position of the marker; or

[0253] If the pre-selected position of the marker is the right side of the corresponding object, and the size of the marker extends beyond the right side of the image frame, then the left side of the object is determined as the target position of the marker; or

[0254] If the pre-selected position of the marker is the left side of the corresponding object, and the size of the marker does not exceed the left side of the image frame, then the left side of the object is determined as the target position of the marker; or

[0255] If the pre-selected position of the identifier is the right side of the corresponding object, and the size of the identifier does not exceed the right side of the image frame, then the right side of the object is determined as the target position of the identifier.

[0256] In some embodiments of this application, the video visualization module 1113 is further used for:

[0257] If the pre-selected position of the identifier is to the left of the corresponding object, and the size of the identifier extends beyond the left side of the image frame, then the target identifier image is drawn to the right side of the object; or

[0258] If the pre-selected position of the identifier is the right side of the corresponding object, and the size of the identifier extends beyond the right side of the image frame, then the target identifier image is drawn to the left side of the object; or

[0259] If the pre-selected position of the identifier is the left side of the corresponding object, and the size of the identifier does not exceed the left side of the image frame, then the target identifier image is drawn to the left side of the object; or

[0260] If the pre-selected position of the identifier is the right side of the corresponding object, and the size of the identifier does not exceed the right side of the image frame, then the target identifier image is drawn to the right side of the object.

[0261] In some embodiments of this application, the inference result further includes a confidence level that the object is of the object type, and the video inference module 1112 is further used for:

[0262] Discard inference results where the confidence level of the object being of the object type is less than a first threshold.

[0263] In some embodiments of this application, the video inference module 1112 is further configured to:

[0264] The inference frequency is determined based on the resolution of the plurality of image frames, wherein the inference frequency means that inference is performed once every X image frames, and the resolution of the image frames is positively correlated with X;

[0265] Inference is performed on the objects in the plurality of image frames according to the inference frequency.

[0266] In some embodiments of this application, the video distribution module 1114 is further used for

[0267] Initiate socket detection of connection requests sent by the general server;

[0268] Upon receiving a connection request from the general server, the data corresponding to the image frame with the added identifier is packetized according to TCP.

[0269] In some embodiments of this application, the video distribution module 1114 is further configured to:

[0270] Configure the size of the socket buffer used for data transmission based on the resolution of the multiple image frames.

[0271] In some embodiments of this application, the image compression module 1122 is further configured to:

[0272] Configure the number of threads used for image compression based on the resolution of the image frame;

[0273] The image frame is compressed using the specified number of threads.

[0274] It should be noted that the functions of each module in the object recognition system 1100 in this application embodiment can be referred to in the above method embodiments. Figures 3 to 10 The specific implementation methods of any embodiment will not be described in detail here.

[0275] Therefore, the object recognition system based on this application increases the concurrency of the inference server by offloading video inference and video compression operations to GPU processors and general-purpose processors. Furthermore, offloading video compression from the GPU server allows full utilization of idle general-purpose servers to complete the video compression operation, eliminating the need for expensive GPU resources for video transcoding and reducing the cost of the object recognition system. Moreover, by analyzing the video characteristics of objects and setting the number of image frames used for inference, real-time recognition of objects in high-definition video can be achieved. Furthermore, by employing image fusion for logo rendering, the visualization effect is improved. This logo can be customized according to user needs, providing support for adaptation to customer systems and enhancing the user experience.

[0276] Each module in the aforementioned object recognition system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0277] Figure 13 This is another schematic structural diagram of the object recognition system provided in the embodiments of this application, such as... Figure 13As shown, the object recognition system 1200 may include: a communication interface 1201, a memory 1202, a processor 1203, a processor 1205, and a communication bus 1204. The communication interface 1201, memory 1202, processor 1203, and processor 1205 communicate with each other via the communication bus 1204. The communication interface 1201 is used for data communication between the object recognition system 1200 and external devices. The memory 1202 can be used to store software programs and modules. The processors 1203 and 1205 execute the software programs and modules stored in the memory 1202, such as the software programs for the corresponding operations in the aforementioned method embodiments.

[0278] In some embodiments, processor 1203 can be a GPU processor and processor 1205 can be a general-purpose processor. Processor 1203 and processor 1205 each correspond to independent memory. Processor 1203 and processor 1205 execute the operations corresponding to the GPU server and general-purpose server in the foregoing embodiments by running the software programs and modules in the corresponding memory.

[0279] For example, the processor 1203 can call software programs and modules stored in the memory to perform the following operations: acquire a video stream acquired by a video acquisition device, parse the video stream into multiple image frames; use an object recognition model to infer the objects in the multiple image frames, and obtain the inference results of the objects in the multiple image frames, wherein the object recognition module is obtained using standard model transfer learning, and the inference results include the object type of the objects in the image frames; add an object type identifier to the image frames according to the object inference results; encapsulate the data corresponding to the image frames with added identifiers according to the Transmission Control Protocol (TCP), and send the encapsulated data to the processor 1205.

[0280] For example, the processor 1205 can decode the received packet data into image frames corresponding to the data, and compress the image frames to convert them into a target video stream.

[0281] In some embodiments, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0282] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the object recognition method of this application embodiment; for brevity, these will not be elaborated further here.

[0283] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the object recognition method of this application embodiment. For simplicity, further details are omitted here.

[0284] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the object recognition method of this application embodiment. For simplicity, further details are omitted here.

[0285] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above may be a general-purpose server, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose server may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0286] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0287] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0288] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0289] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0290] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0291] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0292] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0293] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0294] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of object recognition, characterized by, The application is applied to a real-time identification scene of a water ship body, wherein an object identification system comprises a GPU server and a general server, and the method comprises the following steps: The GPU server acquires a video stream of the water ship body collected by a video collection device, parses the video stream into a plurality of image frames, and uses an object identification model to infer objects in the plurality of image frames to obtain an inference result of the objects in the plurality of image frames, including determining an inference frequency according to resolutions of the plurality of image frames, wherein the inference frequency represents that inference is performed once every X image frames, the resolution of the image frame is positively correlated with X, and the objects in the plurality of image frames are inferred according to the inference frequency; wherein the object identification model is obtained by using standard model transfer learning, the inference result comprises an object type of the object in the image frame, the object identification model is a ship body identification model, and the object is a ship body; an identification of the object type is added in the image frame according to the inference result of the object, the identification is a customized identification; data corresponding to the image frame to which the identification is added is packetized according to a transmission control protocol (TCP), and the packetized data is sent to the general server; The general server decodes the received packetized data into an image frame corresponding to the data, and compresses and converts the image frame into a target video stream; The inference result further comprises a position coordinate of the object, and the identification of the object type is added in the image frame according to the inference result of the object, including drawing an object frame according to the position coordinate of the object, determining a target position of the identification of the object type according to a position of the object frame and a size of the identification of the object type, taking a transparent image of a specific color as a background image, fusing the background image and the identification to obtain a target identification image, and drawing the target identification image to the target position of the identification. The step of determining the target position of the object type identifier based on the position of the object frame and the size of the object type identifier includes: determining a pre-selected position of the object type identifier based on the position of the object frame; determining the target position of the object type identifier based on the pre-selected position of the object type identifier and the size of the object type identifier, wherein, if the pre-selected position of the identifier is the left side of the corresponding object and the size of the identifier extends beyond the left side of the image frame, the right side of the object is determined as the target position of the identifier; or if the pre-selected position of the identifier is the right side of the corresponding object and the size of the identifier extends beyond the right side of the image frame, the left side of the object is determined as the target position of the identifier; or if the pre-selected position of the identifier is the left side of the corresponding object and the size of the identifier does not extend beyond the left side of the image frame, the left side of the object is determined as the target position of the identifier; or if the pre-selected position of the identifier is the right side of the corresponding object and the size of the identifier does not extend beyond the right side of the image frame, the right side of the object is determined as the target position of the identifier.

2. The method of claim 1, wherein, The step of drawing the target identifier image onto the target location of the identifier includes: If the pre-selected position of the identifier is the left side of the corresponding object, and the size of the identifier extends beyond the left side of the image frame, the target identifier image is drawn to the right side of the object; or If the pre-selected position of the identifier is the right side of the corresponding object, and the size of the identifier extends beyond the right side of the image frame, the target identifier image is drawn to the left side of the object; or If the pre-selected position of the identifier is the left side of the corresponding object, and the size of the identifier does not exceed the left side of the image frame, then the target identifier image is drawn to the left side of the object; or If the pre-selected position of the identifier is the right side of the corresponding object, and the size of the identifier does not exceed the right side of the image frame, the target identifier image is drawn to the right side of the object.

3. The method according to any one of claims 1-2, characterized in that, The inference result also includes a confidence level that the object is of the object type, and the method further includes: Discard inference results where the confidence level of the object being of the object type is less than a first threshold.

4. The method according to any one of claims 1-2, characterized in that, The method further includes: The GPU server initiates socket detection to detect connection requests sent by the general server; The process of packetizing the data corresponding to the image frame with the added identifier according to the Transmission Control Protocol (TCP) includes: Upon receiving a connection request from the general server, the data corresponding to the image frame with the added identifier is packetized according to TCP.

5. The method of claim 1, wherein, The method further includes: The GPU server configures the size of the socket buffer for data transmission based on the resolution of the plurality of image frames.

6. The method of claim 1, wherein, The step of compressing the image frame and converting it into a target video stream includes: Configure the number of threads used for image compression based on the resolution of the image frame; The image frame is compressed using the specified number of threads.

7. An object recognition system characterized by, Applications include real-time identification of ship hulls on water, including: GPU server and general server; The GPU server comprises an acquisition module, a video inference module, a video visualization module and a video distribution module. The acquisition module is configured to acquire a video stream collected by a video collection device for a water-borne hull, and parse the video stream into a plurality of image frames. The video inference module is configured to infer objects in the plurality of image frames by using an object recognition model to obtain an inference result of the objects in the plurality of image frames, including determining an inference frequency according to resolutions of the plurality of image frames, wherein the inference frequency represents that inference is performed once every X image frames, the resolution of the image frames is positively correlated with X, and the objects in the plurality of image frames are inferred according to the inference frequency; wherein the object recognition model is obtained by using standard model transfer learning, the inference result includes an object type of the objects in the image frames, the object recognition model is a hull recognition model, and the objects are hulls. The video visualization module is configured to add an identification of the object type in the image frames according to the inference result of the objects, and the identification is a customized identification. The video distribution module is configured to packetize data corresponding to the image frames with the added identification according to a transmission control protocol (TCP), and send the packetized data to the general server. The general server comprises a data decoding module and an image compression module. The data decoding module is configured to decode the received packetized data into image frames corresponding to the data. The image compression module converts the image frames into a target video stream by compression. The inference result further includes position coordinates of the objects, and the video visualization module is specifically configured to draw an object frame according to the position coordinates of the objects, determine a target position of the identification of the object type according to a position of the object frame and a size of the identification of the object type, take a transparent image of a specific color as a background image, fuse the background image and the identification to obtain a target identification image, and draw the target identification image to the target position of the identification. The video visualization module is further configured to: determine a preselected position of the object type identifier according to the position of the object frame; determine a target position of the object type identifier according to the preselected position of the object type identifier and the size of the object type identifier; wherein if the preselected position of the identifier is a left position of the corresponding object and the size of the identifier exceeds the left side of the image frame, the right position of the object is determined as the target position of the identifier; or if the preselected position of the identifier is a right position of the corresponding object and the size of the identifier exceeds the right side of the image frame, the left position of the object is determined as the target position of the identifier; or if the preselected position of the identifier is a left position of the corresponding object and the size of the identifier does not exceed the left side of the image frame, the left position of the object is determined as the target position of the identifier; or if the preselected position of the identifier is a right position of the corresponding object and the size of the identifier does not exceed the right side of the image frame, the right position of the object is determined as the target position of the identifier.

8. An object recognition system, characterized by comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of any one of claims 1 to 6 via executing the executable instructions.

9. A storage medium, characterized by A computer program for storing, the computer program causing a computer to perform the method of any one of claims 1 to 6. A computer program for storing, the computer program causing a computer to perform the method of any one of claims 1 to 6.

Images