Fishing port image detection method and device and storage medium

Abstract

The application discloses a fishing port image detection method and device and a storage medium. It relates to the field of artificial intelligence. The method comprises the following steps: receiving an image detection request corresponding to a fishing port scene, extracting an image address to be detected and request parameters from the image detection request, determining target image data according to the image address, determining a first detection model from a model cache dictionary, and performing inference on the target image data by using the first detection model to obtain a first detection result; if the first detection result indicates that the number of effective fishing boats in the target image data is greater than zero, a plurality of second detection models are determined from the model cache dictionary according to the request parameters, and the target image data is analyzed by using the plurality of second detection models to obtain a second detection result; and the first detection result and the second detection result are fused to obtain a target detection result. Through the application, the problem of high image detection delay of the fishing port in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a method, apparatus, and storage medium for detecting images of fishing ports. Background Technology

[0002] With the deepening of smart fishing port construction, image analysis technology based on computer vision has become a core means of fishing port supervision. In fishing port scenarios, shore-based monitoring cameras, drones, and law enforcement vessels collect large amounts of video streams and image data in real time, aiming to monitor fishing vessels entering and leaving the port.

[0003] In related technologies, image detection tasks based on fishing port scenarios still suffer from technical shortcomings such as high response latency and low resource utilization when facing complex fishing port environments, making it difficult to meet the real-time detection requirements under high concurrency. Specifically, existing technologies mostly adopt a single-request model loading method or a simple encapsulation approach, resulting in the need to repeatedly load model weights before each inference, leading to significant cold start latency. Simultaneously, executing the entire detection chain in targetless or noisy scenarios causes invalid computation and input / output blockage. Furthermore, in scenarios involving multiple people and boats, existing technologies often process modules such as personnel detection, boat license plate recognition, and feature retrieval sequentially, and the output files are written to disk sequentially, further exacerbating end-to-end processing latency. These shortcomings cause fishing port image detection to easily encounter problems such as response timeouts, data backlogs, and high latency when dealing with sudden peak traffic.

[0004] There is currently no effective solution to the problem of high latency in image detection of fishing ports in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, and storage medium for detecting images of fishing ports, in order to solve the problem of high latency in image detection of fishing ports in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a method for detecting images of fishing ports is provided. The method includes: receiving an image detection request corresponding to a fishing port scene, extracting the address of the image to be detected and request parameters from the image detection request, and determining target image data based on the image address; determining a first detection model from a model cache dictionary, and using the first detection model to infer the target image data to obtain a first detection result; if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, determining multiple second detection models from the model cache dictionary based on the request parameters, and using the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result; and performing a fusion process based on the first detection result and the second detection result to obtain a target detection result.

[0007] Furthermore, before receiving the image detection request corresponding to the fishing port scene, the method further includes: determining multiple models to be loaded for processing the image detection request; initializing a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, the key name of the model information is the model path, and the key value of the model information is the model instance; executing a model loading process based on the multiple models to be loaded to generate an instance of each model to be loaded; and updating the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded to obtain a model cache dictionary.

[0008] Further, the first detection result is obtained by reasoning on the target image data using the first detection model, which includes: inputting the target image data into the first detection model and outputting multiple initial candidate boxes using the first detection model; performing non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; calculating the area of ​​each initial fishing boat box to obtain multiple areas, and determining the maximum area from the multiple areas; determining the ratio of each area to the maximum area, and filtering out initial fishing boat boxes with ratios less than the relative area ratio threshold from the multiple initial fishing boat boxes based on the multiple areas to obtain a valid fishing boat set; counting the number of valid fishing boats based on the valid fishing boat set, and determining the first detection result based on the number of valid fishing boats and the valid fishing boat set.

[0009] Furthermore, the multiple second detection models include at least a personnel detection model, a life jacket detection model, and a fishing vessel identification model. These multiple second detection models are used to perform enhanced analysis on the target image data, yielding the following second detection results: using the personnel detection model to determine personnel detection boxes in the target image data, and using the life jacket detection model to determine life jacket detection boxes in the target image data; based on the personnel detection boxes and life jacket detection boxes, determining the information of personnel wearing life jackets and those not wearing life jackets in the target image data; using the fishing vessel identification model to determine the fishing vessel identification information of each valid fishing vessel in the target image data; and based on the information of personnel wearing life jackets, those not wearing life jackets, and the fishing vessel identification information, determining the second detection result.

[0010] Furthermore, determining the fishing vessel identity information of each valid fishing vessel in the target image data using the fishing vessel identity recognition model includes: extracting a set of valid fishing vessels from the first detection result and determining the fishing vessel detection box for each valid fishing vessel in the set; cropping the target image data based on the fishing vessel detection boxes to obtain multiple fishing vessel regions of interest (ROI) data; stacking the multiple fishing vessel ROI data into a batch tensor and determining the batch feature matrix of the batch tensor based on the fishing vessel identity recognition model; calculating the cosine distance between the batch feature matrix and the feature matrix of a pre-set image library, and determining the fishing vessel identity information of each valid fishing vessel based on the cosine distance calculation result.

[0011] Furthermore, after using the first detection model to infer the target image data and obtain the first detection result, the method further includes: if the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, then skip the enhancement analysis process, store the target image data in the result image directory, and determine the first detection result as the target detection result.

[0012] Furthermore, after fusing the first and second detection results to obtain the target detection result, the method further includes: identifying the target fishing vessel in the target image data where the number of people is not zero, and acquiring the target region of interest image of the target fishing vessel, marking the life jacket wearing status of each person in the target region of interest image to obtain a detailed annotation image; and performing parallel disking of the target image data, target detection result, and detailed annotation image to obtain target response information.

[0013] To achieve the above objectives, according to another aspect of this application, a fishing port image detection apparatus is provided. The apparatus includes: a receiving unit, configured to receive an image detection request corresponding to a fishing port scene, extract the address of the image to be detected and request parameters from the image detection request, and determine target image data based on the image address; a first detection unit, configured to determine a first detection model from a model cache dictionary, and use the first detection model to infer the target image data to obtain a first detection result; a second detection unit, configured to, if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, determine multiple second detection models from the model cache dictionary based on the request parameters, and use the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result; and a fusion unit, configured to perform fusion processing based on the first detection result and the second detection result to obtain a target detection result.

[0014] Furthermore, the device also includes: a determining unit, used to determine multiple models to be loaded for processing the image detection request before receiving the image detection request corresponding to the fishing port scene; an initialization unit, used to initialize a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, the key name of the model information is the model path, and the key value of the model information is the model instance; an execution unit, used to execute the model loading process according to the multiple models to be loaded, and generate an instance of each model to be loaded; and an update unit, used to update the key-value pair dictionary according to the multiple models to be loaded and the instance of each model to be loaded, to obtain a model cache dictionary.

[0015] Further, the first detection unit includes: a first input module, used to input target image data into a first detection model and output multiple initial candidate boxes using the first detection model; a non-maximum suppression module, used to perform non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; an area calculation module, used to calculate the area of ​​each initial fishing boat box to obtain multiple areas, and determine the maximum area from the multiple areas; a filtering module, used to determine the ratio of each area to the maximum area, and filter out initial fishing boat boxes whose ratio is less than the relative area ratio threshold from the multiple initial fishing boat boxes based on the multiple areas, to obtain a set of effective fishing boats; and a statistics module, used to count the number of effective fishing boats based on the set of effective fishing boats, and determine the first detection result based on the number of effective fishing boats and the set of effective fishing boats.

[0016] Furthermore, the multiple second detection models include at least a personnel detection model, a life jacket detection model, and a fishing vessel identification model. The second detection unit includes: a detection box determination module, used to determine personnel detection boxes in the target image data using the personnel detection model and life jacket detection boxes in the target image data using the life jacket detection model; an information determination module, used to determine the information of personnel wearing life jackets and personnel not wearing life jackets in the target image data based on the personnel detection boxes and life jacket detection boxes; an identity determination module, used to determine the fishing vessel identity information of each valid fishing vessel in the target image data using the fishing vessel identification model; and a result determination module, used to determine the second detection result based on the information of personnel wearing life jackets, personnel not wearing life jackets, and fishing vessel identity information.

[0017] Furthermore, the identity determination module includes: an extraction submodule, used to extract a set of valid fishing vessels from the first detection result and determine the fishing vessel detection box for each valid fishing vessel in the set; a cropping submodule, used to crop the target image data according to the fishing vessel detection boxes to obtain multiple fishing vessel region of interest data; a stacking submodule, used to stack multiple fishing vessel region of interest data into a batch tensor and determine the batch feature matrix of the batch tensor according to the fishing vessel identity recognition model; and a distance calculation submodule, used to perform cosine distance calculation between the batch feature matrix and the preset image library feature matrix and determine the fishing vessel identity information of each valid fishing vessel according to the cosine distance calculation result.

[0018] Furthermore, the device also includes a skipping unit, which, after inferring the target image data using the first detection model and obtaining the first detection result, if the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, skips the enhancement analysis process, stores the target image data in the result image directory, and determines the first detection result as the target detection result.

[0019] Furthermore, the device also includes: a detail map acquisition unit, used to determine the target fishing vessel with a non-zero number of personnel in the target image data after fusing the first detection result and the second detection result to obtain the target detection result, and to acquire the target region of interest image of the target fishing vessel, and to mark the life jacket wearing status of each person in the target region of interest image to obtain a marked detail map; and a parallel disk dropping unit, used to perform parallel disk dropping on the target image data, the target detection result and the marked detail map to obtain target response information.

[0020] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any fishing port image detection method.

[0021] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any fishing port image detection method.

[0022] According to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the fishing port image detection method described above.

[0023] In this embodiment, an image detection request corresponding to a fishing port scene is received, and the address of the image to be detected and the request parameters are extracted from the image detection request. The target image data is determined based on the image address. A first detection model is determined from the model cache dictionary, and the first detection model is used to infer the target image data to obtain a first detection result. If the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, multiple second detection models are determined from the model cache dictionary based on the request parameters, and the multiple second detection models are used to perform enhancement analysis on the target image data to obtain a second detection result. The first detection result and the second detection result are fused to obtain the target detection result, thus solving the technical problem of high image detection latency in fishing ports in the prior art. Attached Figure Description

[0024] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 A hardware block diagram of a computer terminal for implementing a fishing port image detection method is shown.

[0026] Figure 2 This is a flowchart of a fishing port image detection method provided according to an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of a fishing port image detection device provided according to an embodiment of this application;

[0028] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, do not violate public order and good morals, and provide corresponding access points for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0032] Example 1

[0033] According to an embodiment of this application, a method embodiment for detecting fishing port images is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a fishing port image detection method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fishing port image detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned fishing port image detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0038] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0039] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for detecting images of fishing ports is shown. Figure 2 This is a flowchart of a fishing port image detection method according to Embodiment 1 of this application.

[0040] Step S201: Receive the image detection request corresponding to the fishing port scene, extract the image address to be detected and the request parameters from the image detection request, and determine the target image data based on the image address.

[0041] Optionally, an image detection request sent from a client can be received via a program interface. The image detection request may include a field about the image address (e.g., the image_url field), which may represent a local file path or a network link. Based on the image address field, the target image data can be read from the local file path or downloaded as a binary stream from the network link.

[0042] Optionally, the request parameters are used to dynamically control the detection strategy. The request parameters may include a detection confidence threshold (e.g., 0.5), parameters for whether to enable the re-identification model (e.g., enable), whether to enable ship plate optical character recognition, relative area ratio threshold, etc.

[0043] Step S202: Determine the first detection model from the model cache dictionary, and use the first detection model to infer the target image data to obtain the first detection result.

[0044] Optionally, the model cache dictionary can be a globally shared dictionary obtained during service startup (existing only in memory and not persistently stored) used to implement singleton reuse, ensuring that the same model is loaded only once and avoiding repeated loading for each request. It stores model information in key-value pair format, where the key is the model path and the key is the model instance. The model cache dictionary guides the one-time loading of the required models during image detection, thus avoiding repeated model loading. The model cache dictionary includes at least model information about the first detection model, which can be a target detection model for fishing boats. This first detection model can be used to infer the target image data to obtain the first detection result, which is the set of valid fishing boat targets after preliminary screening. This set does not include information such as fishing boat identity or personnel. The first detection result is used to quickly determine whether there are valid fishing boats in the target image data (i.e., whether the number of valid fishing boats is greater than zero), providing a basis for the startup control of subsequent augmentation analysis.

[0045] Step S203: If the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, then multiple second detection models are determined from the model cache dictionary according to the request parameters, and the target image data is enhanced by using the multiple second detection models to obtain the second detection result.

[0046] Optionally, if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, then multiple second detection models to be used are determined from the model cache dictionary according to the request parameters, and an augmentation analysis process is executed to obtain the second detection result. The second detection model can be a personnel detection model, a life jacket detection model, or a fishing boat identification model, etc., used for augmentation analysis. If the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, then the augmentation analysis process can be skipped directly, and the first detection result can be directly determined as the target detection result, thereby skipping the time-consuming and high-overhead detection and achieving millisecond-level response in high-concurrency fishing boat scene image detection.

[0047] Step S204: Based on the first detection result and the second detection result, perform fusion processing to obtain the target detection result.

[0048] Optionally, the target detection result can be a JSON (a data format) object. The first detection result and the second detection result are fused. This can be done by reading information on people wearing life jackets (e.g., the location information of the detection box), people not wearing life jackets (e.g., the location information of the detection box), and fishing vessel identity information from the second detection result. The second detection result is then compared with the fishing vessel frames of each valid fishing vessel in the valid fishing vessel set of the first detection result (e.g., if the geometric center of the personnel detection box is located in the fishing vessel frame, then the number of people in that fishing vessel frame is incremented by 1). This determines the fishing vessel identity, the number of people wearing life jackets, and the number of people not wearing life jackets for each valid fishing vessel. The fishing vessel identity, the number of people wearing life jackets, the number of people not wearing life jackets, and the coordinate information of all detection boxes (e.g., fishing vessel frames, personnel detection boxes, and life jacket detection boxes) are stored as a JSON object to obtain the target detection result.

[0049] In summary, by obtaining the first detection result based on the first detection model, and performing enhancement analysis on the target image data based on multiple second detection models only when the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, the frequent application of time-consuming tasks is avoided, and the input load of enhancement analysis is reduced. At the same time, by determining the multiple second detection models to be used through request parameters and model cache dictionary, the repeated loading of models is avoided, further reducing resource consumption. Therefore, this solution reduces the image detection latency of fishing ports.

[0050] To reduce image detection latency in fishing ports, the method may optionally include the following steps before receiving image detection requests for fishing port scenes: determining multiple models to be loaded for processing image detection requests; initializing a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, where the key name of the model information is the model path and the key value is the model instance; executing a model loading process based on the multiple models to be loaded to generate an instance of each model to be loaded; and updating the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded to obtain a model cache dictionary.

[0051] Optionally, a preset configuration file can be read, which predefines all models to be loaded for image detection in a fishing port scenario (e.g., the first detection model). The first detection model is a mandatory model. The configuration file can determine whether other second detection models (e.g., life jacket detection model, ship license plate optical character recognition model) need to be loaded. A globally unique hash table can be initialized in memory as a key-value dictionary. The model path in the initialized key-value dictionary represents the model name and absolute path. Initial key values ​​can be placeholders to reserve and store subsequently loaded model instances. A model loading process can be executed based on the models to be loaded, generating an instance of each model to be loaded and updating the model information in the key-value dictionary to obtain a model cache dictionary. Alternatively, after each image detection request is received, the model cache dictionary can be checked to see if all the key values ​​of the second detection models indicated by the request parameters already exist. If not, the missing second detection models are loaded according to the request parameters, and the model cache dictionary is updated after loading the models. After the model is instantiated, the model cache dictionary contains memory-mapped instances of all required models. At this point, the image detection service in the fishing port scenario is ready.

[0052] In summary, by identifying multiple models to be loaded for processing image detection requests; initializing a key-value pair dictionary, where the key name of the model information is the model path and the key value is the model instance; executing the model loading process based on the multiple models to be loaded to generate an instance of each model to be loaded; and updating the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded to obtain a model cache dictionary, the system avoids repeated model loading, avoids multiple threads competing for resources under high concurrency, and eliminates the need for repeated loading before each inference, thus solving the problem of high cold start latency and reducing the image detection latency in the fishing port.

[0053] To reduce the image detection latency in fishing ports, optionally, the first detection model is used to infer the target image data to obtain a first detection result, including: inputting the target image data into the first detection model and outputting multiple initial candidate boxes using the first detection model; performing non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; calculating the area of ​​each initial fishing boat box to obtain multiple areas, and determining the largest area from the multiple areas; determining the ratio of each area to the largest area, and filtering out initial fishing boat boxes with ratios less than a relative area ratio threshold from the multiple initial fishing boat boxes based on the multiple areas to obtain a valid fishing boat set; counting the number of valid fishing boats based on the valid fishing boat set, and determining the first detection result based on the number of valid fishing boats and the valid fishing boat set.

[0054] Optionally, the first detection model can be a model specifically designed for fishing vessel detection, trained based on an open-source object detection model (e.g., the YOLO model). The first detection model outputs multiple bounding boxes and corresponding confidence scores. These confidence scores can be filtered according to a detection confidence threshold in the request parameters to obtain multiple initial candidate boxes. Each initial candidate box includes coordinate information, a category label (e.g., fishing vessel), and a confidence score. The overlap threshold for non-maximum suppression can be 0.5 to remove duplicate boxes, resulting in multiple initial fishing vessel boxes. The coordinate information of each initial fishing vessel box is traversed, and the area of ​​each initial fishing vessel box is calculated. For example, the area calculation can be based on the following formula:

[0055] area_i=(x_max_i-x_min_i)×(y_max_i-y_min_i)

[0056] Here, area_i represents the area of ​​the i-th initial fishing boat frame, and x_max_i, x_min_i, y_max_i, and y_min_i represent the coordinates of the four corners of the i-th initial fishing boat frame. After obtaining the areas of all initial fishing boat frames, the largest area is found, and the ratio of the area of ​​each initial fishing boat frame to the largest area is calculated. The relative area ratio threshold can be data determined from the request parameters; for example, the relative area ratio threshold can be 0.1. Initial fishing boat frames whose area ratio is less than the relative area ratio threshold (real fishing boats are larger than noise targets in a single frame image) can be filtered out from all initial fishing boat frames. The remaining initial fishing boat frames are used as valid fishing boat frames to construct a valid fishing boat set. The number of valid fishing boat frames in the valid fishing boat set is counted to obtain the number of valid fishing boats, thus obtaining the first detection result.

[0057] In summary, by inputting the target image data into the first detection model, the first detection model outputs multiple initial candidate boxes; non-maximum suppression is performed on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; the area of ​​each initial fishing boat box is calculated to obtain multiple areas, and the largest area is determined from the multiple areas; the ratio of each area to the largest area is determined, and initial fishing boat boxes with ratios less than the relative area ratio threshold are filtered out from the multiple initial fishing boat boxes based on the multiple areas to obtain a set of effective fishing boats; the number of effective fishing boats is counted based on the set of effective fishing boats, and the first detection result is determined based on the number of effective fishing boats and the set of effective fishing boats. This method quickly filters out noisy targets, reduces the input load of subsequent enhancement analysis, and reduces the image detection latency of fishing ports.

[0058] To reduce the image detection latency in fishing ports, optionally, multiple second detection models include at least a personnel detection model, a life jacket detection model, and a fishing vessel identification model. These multiple second detection models are used to perform enhanced analysis on the target image data, yielding second detection results including: using the personnel detection model to determine personnel detection boxes in the target image data, and using the life jacket detection model to determine life jacket detection boxes in the target image data; based on the personnel detection boxes and life jacket detection boxes, determining the information of personnel wearing life jackets and those not wearing life jackets in the target image data; using the fishing vessel identification model to determine the fishing vessel identification information of each valid fishing vessel in the target image data; and based on the information of personnel wearing life jackets, those not wearing life jackets, and the fishing vessel identification information, determining the second detection result.

[0059] Optionally, the loaded personnel detection model, life jacket detection model, and fishing vessel identification model can be determined based on the model cache dictionary. The personnel detection model and life jacket detection model can be models trained on open-source object detection models (e.g., the personnel detection dataset has personnel bounding boxes labeled; while the life jacket detection model has life jacket bounding boxes labeled, but not associated with personnel) based on the corresponding labeled datasets (e.g., the personnel detection dataset has personnel bounding boxes labeled; while the life jacket detection model has life jacket bounding boxes labeled, but not associated with personnel). The fishing vessel identification model can be used to determine the fishing vessel identification information of each valid fishing vessel in the target image data (e.g., determining the ID information of the known fishing vessels to which the valid fishing vessel belongs). The personnel detection model can be used to determine the personnel detection boxes in the target image data, and the life jacket detection model can be used to determine the life jacket detection boxes in the target image data. Based on the personnel detection boxes and life jacket detection boxes, determining the information of personnel wearing life jackets and personnel not wearing life jackets in the target image data can be divided into the following two steps:

[0060] The first step is to preliminarily determine whether a person is wearing a life jacket. If the geometric center of the life jacket detection frame is located within the person detection frame, it means that the person is wearing a life jacket; otherwise, the person is not wearing a life jacket.

[0061] The second step is to perform obfuscation suppression processing based on the first step. For the detection box of a person who is not wearing a life jacket, if the intersection-union ratio of the detection box of the person and any life jacket detection box is greater than a preset threshold (for example, 0.7), the life jacket wearing status of the person corresponding to the detection box of the person is corrected to that of the person who is wearing a life jacket.

[0062] Optionally, if the request parameters specify enabling optical character recognition (OCR) for vessel license plates, the second detection model also includes an OCR model. This model can determine the vessel license plate area from the detection frame of a valid fishing vessel and identify the vessel license plate characters within that area using the OCR model. The identified characters are then compared with the fishing vessel's identity information. If they match, the fishing vessel's identity information is confirmed to be correct; otherwise, the target image data is sent to the target server for manual verification.

[0063] In summary, by using a personnel detection model to determine personnel detection boxes in the target image data, and a life jacket detection model to determine life jacket detection boxes in the target image data, the information of personnel wearing life jackets and those not wearing life jackets in the target image data is determined based on the personnel detection boxes and life jacket detection boxes. A fishing vessel identification model is used to determine the fishing vessel identity information of each valid fishing vessel in the target image data. Based on the information of personnel wearing life jackets, those not wearing life jackets, and the fishing vessel identity information, the second detection result is determined. This decoupling of different enhancement analyses allows multiple different second detection models to be executed in parallel, reducing the image detection latency in fishing ports.

[0064] To reduce the image detection latency in fishing ports, optionally, the method of using a fishing vessel identification model to determine the fishing vessel identification information of each valid fishing vessel in the target image data includes: extracting a set of valid fishing vessels from the first detection result and determining a fishing vessel detection box for each valid fishing vessel in the set; cropping the target image data based on the fishing vessel detection boxes to obtain multiple fishing vessel regions of interest (ROI) data; stacking the multiple fishing vessel ROI data into a batch tensor and determining the batch feature matrix of the batch tensor based on the fishing vessel identification model; calculating the cosine distance between the batch feature matrix and a pre-set image library feature matrix, and determining the fishing vessel identification information of each valid fishing vessel based on the cosine distance calculation result.

[0065] Optionally, the valid fishing vessel bounding boxes in the valid fishing vessel set can be determined as the fishing vessel detection boxes for valid fishing vessels. The corresponding regions can be cropped from the target image data based on the coordinate information of the fishing vessel detection boxes. Alternatively, the horizontal and vertical coordinates of the fishing vessel detection boxes can be extended outwards by 10% to form the region of interest (ROI) data for the fishing vessels. All ROI data can be uniformly scaled to a preset size (e.g., 256 pixels wide and 128 pixels high). Tensors from multiple ROI data are stacked into batches to obtain batch tensors (including feature vectors of multiple query targets). These batch tensors are then fed into a fishing vessel identification model (e.g., a re-identification model, i.e., a re-identification feature extraction network) to obtain the batch feature matrix F∈R. N×d(Where N is the number of effective fishing vessels, and d is the dimension of the feature vector, for example, it can be 512-dimensional). The offline constructed pre-built image library feature matrix is ​​G∈R. M×d (M is the total number of samples in the image library, i.e., the number of features of all known fishing boat sample images), and record the fishing boat label_j (i.e., the fishing boat ID corresponding to the j-th image library sample) to which each feature belongs. Calculate the pairwise cosine distance matrix D∈R between the batch feature matrix and the preset image library feature matrix. N×M The formula is:

[0066] D[i][j]=1-(F[i] G[j]) / (||F[i]|| ||G[j]||)

[0067] Where F[i] represents the feature vector of the i-th query target, G[j] represents the feature vector of the j-th image sample, || || represents the L2 norm. For each query target i (i=1,2,…,N), for each label ( (Representing a fishing vessel ID), calculate the minimum distance of all samples under that label:

[0068] d_i( =min_{j:label_j= }D[i][j]

[0069] The label with the smallest distance is used as the identification result pred_i for the fishing vessel's identity information.

[0070] pred_i=argmin_ d_i( )

[0071] Optionally, a distance threshold (e.g., 0.4) can be set. If the minimum distance exceeds the distance threshold, it indicates that the recognition has failed (no known fishing vessel is matched in the image library). In this case, optical character recognition can be triggered, and the fishing vessel's identity information can be determined based on the result of optical character recognition.

[0072] In summary, by extracting a set of valid fishing vessels from the first detection result and determining the detection box for each valid fishing vessel in the set; cropping the target image data based on the detection boxes to obtain multiple regions of interest (ROI) data for fishing vessels; stacking these multiple ROI data into a batch tensor and determining the batch feature matrix of the batch tensor based on the fishing vessel identification model; calculating the cosine distance between the batch feature matrix and the feature matrix of a pre-set image library, and determining the fishing vessel identification information for each valid fishing vessel based on the cosine distance calculation result, this method provides the ability to determine the fishing vessel identification information of multiple valid fishing vessels in parallel, reducing the image detection latency in fishing ports.

[0073] To reduce the image detection latency in fishing ports, optionally, after using the first detection model to infer the target image data and obtain the first detection result, the method further includes: if the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, then skip the enhancement analysis process, store the target image data in the result image directory, and determine the first detection result as the target detection result.

[0074] Optionally, the number of valid fishing boats in the first detection result can be checked. If the number of valid fishing boats is zero, it means that no valid fishing boats were found in the current target image data after noise filtering. In this case, a fast short-circuit process without a target can be entered to complete the processing of this image detection request with extremely low computational overhead. Specifically, no second detection model can be called, skipping time-consuming calculations, and the target image data can be directly stored in the results directory. The results directory can be a hierarchical directory created based on the current date. For example, the results directory can include an original image directory for storing the target image data, a results directory for storing the first detection result, and a labeled image directory for storing all initial fishing boat bounding boxes. To better distinguish empty scenes, a semi-transparent text watermark can also be drawn on the target image data for quick manual identification.

[0075] In summary, by skipping the enhancement analysis process if the first detection result indicates that the number of valid fishing boats in the target image data is zero, storing the target image data in the result image directory, and determining the first detection result as the target detection result, the image detection latency of fishing ports is reduced.

[0076] To reduce the image detection latency in fishing ports, optionally, after fusing the first and second detection results to obtain the target detection result, the method further includes: identifying target fishing vessels with a non-zero number of personnel in the target image data, acquiring a target region of interest image of the target fishing vessel, marking the life jacket wearing status of each person in the target region of interest image to obtain a detailed annotation image; and performing parallel disk write operations on the target image data, target detection results, and detailed annotation image to obtain target response information.

[0077] Optionally, the number of people on each fishing boat can be determined based on the target detection results. Valid fishing boat bounding boxes corresponding to target fishing boats with a non-zero number of people can be obtained. Based on the aforementioned cropping method, a target region of interest (ROI) image can be cropped, and the fishing boat's identity information (e.g., fishing boat ID) can be labeled in the ROI image. For each person detection box, the life jacket wearing status (person wearing life jacket / person not wearing life jacket) can be labeled, resulting in a detailed labeling image. The target image data, target detection results, and detailed labeling image are constructed into three independent asynchronous write tasks and submitted to a thread pool for concurrent execution, achieving parallel disk write. After the write tasks are completed, the total time spent processing the image detection request is recorded. Based on the write paths of the target image data, target detection results, and detailed labeling image, and the total time spent, target response data for the client is generated.

[0078] In summary, by identifying target fishing vessels with a non-zero number of personnel in the target image data and obtaining the target region of interest (ROI) image of the target fishing vessel, and annotating the life jacket wearing status of each person in the ROI image to obtain an annotation detail map, the target image data, target detection results, and annotation detail map are written to disk in parallel to obtain target response information, thereby improving writing efficiency and reducing image detection latency in fishing ports.

[0079] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0080] Example 2

[0081] This application also provides a fishing port image detection device. It should be noted that the fishing port image detection device of this application can be used to execute the fishing port image detection method provided in this application. The fishing port image detection device provided in this application is described below.

[0082] According to an embodiment of this application, an apparatus for implementing the above-described fishing port image detection method is also provided, such as... Figure 3 As shown, the device includes:

[0083] The receiving unit 301 is used to receive an image detection request corresponding to the fishing port scene, extract the image address to be detected and the request parameters from the image detection request, and determine the target image data based on the image address.

[0084] The first detection unit 302 is used to determine a first detection model from the model cache dictionary and use the first detection model to infer the target image data to obtain a first detection result.

[0085] The second detection unit 303 is used to determine multiple second detection models from the model cache dictionary according to the request parameters if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, and to use the multiple second detection models to perform enhancement analysis on the target image data to obtain the second detection result.

[0086] The fusion unit 304 is used to perform fusion processing based on the first detection result and the second detection result to obtain the target detection result.

[0087] The fishing port image detection device provided in this application embodiment receives an image detection request corresponding to a fishing port scene through a receiving unit 301, extracts the image address to be detected and request parameters from the image detection request, determines the target image data based on the image address, determines a first detection model from a model cache dictionary through a first detection unit 302, and uses the first detection model to infer the target image data to obtain a first detection result. If the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, the second detection unit 303 determines multiple second detection models from the model cache dictionary based on the request parameters, and uses the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result. The fusion unit 304 performs fusion processing based on the first detection result and the second detection result to obtain the target detection result. This solves the problem of high image detection latency in fishing ports in related technologies, thereby achieving the effect of reducing image detection latency in fishing ports.

[0088] Optionally, in the fishing port image detection apparatus provided in this application embodiment, the apparatus further includes: a determining unit, configured to determine multiple models to be loaded for processing the image detection request before receiving the image detection request corresponding to the fishing port scene; an initialization unit, configured to initialize a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, the key name of the model information is the model path, and the key value of the model information is the model instance; an execution unit, configured to execute a model loading process based on the multiple models to be loaded, and generate an instance of each model to be loaded; and an update unit, configured to update the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded, to obtain a model cache dictionary.

[0089] Optionally, in the fishing port image detection device provided in this application embodiment, the first detection unit includes: a first input module, used to input target image data into a first detection model and output multiple initial candidate boxes using the first detection model; a non-maximum suppression module, used to perform non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; an area calculation module, used to calculate the area of ​​each initial fishing boat box to obtain multiple areas, and determine the maximum area from the multiple areas; a filtering module, used to determine the ratio of each area to the maximum area, and filter out initial fishing boat boxes whose ratio is less than the relative area ratio threshold from the multiple initial fishing boat boxes according to the multiple areas, to obtain a set of effective fishing boats; and a statistics module, used to count the number of effective fishing boats based on the set of effective fishing boats, and determine the first detection result based on the number of effective fishing boats and the set of effective fishing boats.

[0090] Optionally, in the fishing port image detection device provided in this application embodiment, the multiple second detection models include at least a personnel detection model, a life jacket detection model, and a fishing vessel identification model. The second detection unit includes: a detection box determination module, used to determine personnel detection boxes in the target image data using the personnel detection model and to determine life jacket detection boxes in the target image data using the life jacket detection model; an information determination module, used to determine the information of personnel wearing life jackets and personnel not wearing life jackets in the target image data based on the personnel detection boxes and life jacket detection boxes; an identity determination module, used to determine the fishing vessel identity information of each valid fishing vessel in the target image data using the fishing vessel identification model; and a result determination module, used to determine the second detection result based on the information of personnel wearing life jackets, the information of personnel not wearing life jackets, and the fishing vessel identity information.

[0091] Optionally, in the fishing port image detection device provided in this application embodiment, the identity determination module includes: an extraction submodule, used to extract a set of valid fishing vessels from the first detection result and determine the fishing vessel detection box for each valid fishing vessel in the set of valid fishing vessels; a cropping submodule, used to crop the target image data according to the fishing vessel detection box to obtain multiple fishing vessel region of interest data; a stacking submodule, used to stack the multiple fishing vessel region of interest data into a batch tensor and determine the batch feature matrix of the batch tensor according to the fishing vessel identity recognition model; and a distance calculation submodule, used to perform cosine distance calculation between the batch feature matrix and the preset image library feature matrix and determine the fishing vessel identity information of each valid fishing vessel according to the cosine distance calculation result.

[0092] Optionally, in the fishing port image detection device provided in the embodiments of this application, the device further includes: a skipping unit, which is used to skip the enhancement analysis process and store the target image data in the result image directory and determine the first detection result as the target detection result after the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, after reasoning on the target image data using the first detection model.

[0093] Optionally, in the fishing port image detection device provided in this application embodiment, the device further includes: a detail map acquisition unit, used to determine the target fishing vessel with a non-zero number of personnel in the target image data after performing fusion processing based on the first detection result and the second detection result to obtain the target detection result, and to acquire the target region of interest image of the target fishing vessel, and to mark the life jacket wearing status of each person in the target region of interest image to obtain a marked detail map; and a parallel disk dropping unit, used to perform parallel disk dropping on the target image data, the target detection result and the marked detail map to obtain target response information.

[0094] It should be noted that the receiving unit 301, the first detection unit 302, the second detection unit 303, and the fusion unit 304 mentioned above correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by the units and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of the device and run in the computer terminal 10 provided in Embodiment 1.

[0095] Example 3

[0096] Embodiments of this application may provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0097] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] The processor can access information and applications stored in memory via a transmission device to perform the following steps: receiving an image detection request corresponding to a fishing port scene, extracting the image address to be detected and request parameters from the image detection request, and determining the target image data based on the image address; determining a first detection model from the model cache dictionary, and using the first detection model to infer the target image data to obtain a first detection result; if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, determining multiple second detection models from the model cache dictionary based on the request parameters, and using the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result; and performing fusion processing based on the first detection result and the second detection result to obtain the target detection result.

[0099] The processor can also invoke information and applications stored in memory via a transmission device to perform the following steps: determine multiple models to be loaded for processing image detection requests; initialize a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, where the key name of the model information is the model path and the key value of the model information is the model instance; execute a model loading process based on the multiple models to be loaded to generate an instance of each model to be loaded; update the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded to obtain a model cache dictionary.

[0100] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: inputting the target image data into the first detection model, and using the first detection model to output multiple initial candidate boxes; performing non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; calculating the area of ​​each initial fishing boat box to obtain multiple areas, and determining the maximum area from the multiple areas; determining the ratio of each area to the maximum area, and filtering out initial fishing boat boxes with ratios less than the relative area ratio threshold from the multiple initial fishing boat boxes based on the multiple areas to obtain a set of effective fishing boats; counting the number of effective fishing boats based on the set of effective fishing boats, and determining the first detection result based on the number of effective fishing boats and the set of effective fishing boats.

[0101] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: using a personnel detection model to determine the personnel detection box in the target image data, and using a life jacket detection model to determine the life jacket detection box in the target image data; based on the personnel detection box and the life jacket detection box, determining the information of personnel wearing life jackets and personnel not wearing life jackets in the target image data; using a fishing vessel identification model to determine the fishing vessel identification information of each valid fishing vessel in the target image data; and based on the information of personnel wearing life jackets, personnel not wearing life jackets, and fishing vessel identification information, determining the second detection result.

[0102] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: extract the set of valid fishing vessels from the first detection result and determine the fishing vessel detection box for each valid fishing vessel in the set; crop the target image data according to the fishing vessel detection box to obtain multiple fishing vessel region of interest data; stack the multiple fishing vessel region of interest data into a batch tensor and determine the batch feature matrix of the batch tensor according to the fishing vessel identity recognition model; calculate the cosine distance between the batch feature matrix and the preset image library feature matrix, and determine the fishing vessel identity information of each valid fishing vessel according to the cosine distance calculation result.

[0103] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: if the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, then skip the enhancement analysis process, store the target image data in the result image directory, and determine the first detection result as the target detection result.

[0104] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: determine the target fishing boat in the target image data where the number of people is not zero, and obtain the target fishing boat's region of interest image, mark the life jacket wearing status of each person in the target region of interest image to obtain a detailed annotation map; perform parallel disk write to the target image data, target detection results and detailed annotation map to obtain target response information.

[0105] This application provides a scheme for detecting images of fishing ports. It receives an image detection request corresponding to a fishing port scene, extracts the address of the image to be detected and request parameters from the request, and determines the target image data based on the image address. A first detection model is determined from a model cache dictionary, and inference is performed on the target image data using the first detection model to obtain a first detection result. If the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, multiple second detection models are determined from the model cache dictionary based on the request parameters, and enhancement analysis is performed on the target image data using these multiple second detection models to obtain a second detection result. Finally, the first and second detection results are fused to obtain the target detection result, thus solving the technical problem of high latency in fishing port image detection in the prior art.

[0106] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0107] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0108] Example 4

[0109] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the fishing port image detection method provided in Embodiment 1.

[0110] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0111] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: receiving an image detection request corresponding to a fishing port scene, extracting the image address to be detected and request parameters from the image detection request, and determining target image data based on the image address; determining a first detection model from a model cache dictionary, and using the first detection model to infer the target image data to obtain a first detection result; if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, determining multiple second detection models from the model cache dictionary based on the request parameters, and using the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result; and performing fusion processing based on the first detection result and the second detection result to obtain a target detection result.

[0112] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: determining multiple models to be loaded for processing image detection requests; initializing a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, the key name of the model information is the model path, and the key value of the model information is the model instance; executing a model loading process based on the multiple models to be loaded to generate an instance of each model to be loaded; updating the key-value pair dictionary based on the multiple models to be loaded and the instance of each model to be loaded to obtain a model cache dictionary.

[0113] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: inputting target image data into a first detection model, and using the first detection model to output multiple initial candidate boxes; performing non-maximum suppression on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; calculating the area of ​​each initial fishing boat box to obtain multiple areas, and determining the maximum area from the multiple areas; determining the ratio of each area to the maximum area, and filtering out initial fishing boat boxes with ratios less than a relative area ratio threshold from the multiple initial fishing boat boxes based on the multiple areas to obtain a valid fishing boat set; counting the number of valid fishing boats based on the valid fishing boat set, and determining a first detection result based on the number of valid fishing boats and the valid fishing boat set.

[0114] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: determining a personnel detection box in the target image data using a personnel detection model, and determining a life jacket detection box in the target image data using a life jacket detection model; determining information on personnel wearing life jackets and personnel not wearing life jackets in the target image data based on the personnel detection box and the life jacket detection box; determining the fishing vessel identity information of each valid fishing vessel in the target image data using a fishing vessel identification model; and determining a second detection result based on the information on personnel wearing life jackets, personnel not wearing life jackets, and fishing vessel identity information.

[0115] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: extracting a set of valid fishing vessels from the first detection result, and determining a fishing vessel detection box for each valid fishing vessel in the set; cropping the target image data according to the fishing vessel detection boxes to obtain multiple fishing vessel region of interest data; stacking the multiple fishing vessel region of interest data into a batch tensor, and determining the batch feature matrix of the batch tensor according to the fishing vessel identity recognition model; calculating the cosine distance between the batch feature matrix and the preset image library feature matrix, and determining the fishing vessel identity information of each valid fishing vessel according to the cosine distance calculation result.

[0116] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: if the first detection result indicates that the number of valid fishing boats in the target image data is equal to zero, then the enhancement analysis process is skipped, the target image data is stored in the result image directory, and the first detection result is determined as the target detection result.

[0117] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for performing the following steps: determining a target fishing vessel in the target image data where the number of people is not zero, and acquiring a target region of interest image of the target fishing vessel, annotating the life jacket wearing status of each person in the target region of interest image to obtain an annotation detail image; performing parallel disking of the target image data, target detection results, and annotation detail image to obtain target response information.

[0118] This application also provides a computer program product, which, when executed on a data processing device, is suitable for performing the steps of a fishing port image detection method.

[0119] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0120] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0125] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting images of fishing ports, characterized in that, include: Receive an image detection request corresponding to a fishing port scene, extract the image address to be detected and request parameters from the image detection request, and determine the target image data based on the image address; A first detection model is determined from the model cache dictionary, and the first detection model is used to infer the target image data to obtain a first detection result; If the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, then multiple second detection models are determined from the model cache dictionary according to the request parameters, and the target image data is enhanced using the multiple second detection models to obtain a second detection result; The target detection result is obtained by fusing the first detection result and the second detection result.

2. The method according to claim 1, characterized in that, Before receiving the image detection request corresponding to the fishing port scene, the method further includes: Identify multiple models to be loaded for processing the image detection request; Initialize a key-value pair dictionary, wherein the key-value pair dictionary is used to store model information in key-value pair form, the key name of the model information is the model path, and the key value of the model information is the model instance; The model loading process is executed based on the multiple models to be loaded, and an instance of each of the models to be loaded is generated; Based on the plurality of models to be loaded and the instance of each model to be loaded, the key-value dictionary is updated to obtain the model cache dictionary.

3. The method according to claim 1, characterized in that, The first detection result obtained by inferring from the target image data using the first detection model includes: The target image data is input into the first detection model, and the first detection model is used to output multiple initial candidate boxes. Non-maximum suppression is performed on the multiple initial candidate boxes to obtain multiple initial fishing boat boxes; Calculate the area of ​​each of the initial fishing boat frames to obtain multiple areas, and determine the largest area from the multiple areas; Determine the ratio of each area to the maximum area, and filter out initial fishing boat frames with ratios less than a relative area ratio threshold from the multiple initial fishing boat frames based on the multiple areas to obtain an effective fishing boat set; The number of valid fishing vessels is counted based on the set of valid fishing vessels, and the first detection result is determined based on the number of valid fishing vessels and the set of valid fishing vessels.

4. The method according to claim 1, characterized in that, The plurality of second detection models include at least a personnel detection model, a life jacket detection model, and a fishing vessel identification model. The target image data is enhanced using these multiple second detection models to obtain the following second detection results: The personnel detection model is used to determine the personnel detection bounding box in the target image data, and the life jacket detection model is used to determine the life jacket detection bounding box in the target image data; Based on the personnel detection frame and the life jacket detection frame, determine the information of personnel wearing life jackets and personnel not wearing life jackets in the target image data; The fishing vessel identification model is used to determine the fishing vessel identification information of each valid fishing vessel in the target image data; The second detection result is determined based on the information of the personnel wearing life jackets, the information of the personnel not wearing life jackets, and the identity information of the fishing vessel.

5. The method according to claim 4, characterized in that, The fishing vessel identification model used to determine the identity information of each valid fishing vessel in the target image data includes: Extract the set of valid fishing vessels from the first detection result, and determine the fishing vessel detection frame for each valid fishing vessel in the set of valid fishing vessels. The target image data is cropped based on the fishing boat detection box to obtain multiple fishing boat region of interest data. The data of the multiple fishing vessel regions of interest are stacked into a batch tensor, and the batch feature matrix of the batch tensor is determined according to the fishing vessel identification model. The batch feature matrix is ​​compared with the preset image library feature matrix by cosine distance calculation, and the fishing vessel identity information of each valid fishing vessel is determined based on the cosine distance calculation result.

6. The method according to claim 1, characterized in that, After using the first detection model to infer the target image data and obtain a first detection result, the method further includes: If the first detection result indicates that the number of valid fishing boats in the target image data is zero, then the enhancement analysis process is skipped, the target image data is stored in the result image directory, and the first detection result is determined as the target detection result.

7. The method according to claim 1, characterized in that, After fusing the first detection result and the second detection result to obtain the target detection result, the method further includes: Identify the target fishing vessel in the target image data where the number of people is not zero, and obtain the target region of interest image of the target fishing vessel. Mark the life jacket wearing status of each person in the target region of interest image to obtain a detailed annotation image. Parallel disk write operations are performed on the target image data, the target detection results, and the annotation detail map to obtain target response information.

8. A fishing port image detection device, characterized in that, include: The receiving unit is used to receive an image detection request corresponding to the fishing port scene, extract the image address to be detected and the request parameters from the image detection request, and determine the target image data based on the image address. The first detection unit is used to determine a first detection model from the model cache dictionary and use the first detection model to infer the target image data to obtain a first detection result; The second detection unit is configured to, if the first detection result indicates that the number of valid fishing boats in the target image data is greater than zero, determine multiple second detection models from the model cache dictionary according to the request parameters, and use the multiple second detection models to perform enhancement analysis on the target image data to obtain a second detection result; The fusion unit is used to perform fusion processing based on the first detection result and the second detection result to obtain the target detection result.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the fishing port image detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the fishing port image detection method according to any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the fishing port image detection method according to any one of claims 1 to 7.

Images