A deep-sea sediment sampling system and method based on image recognition technology

Abstract

This invention relates to deep-sea sampling technology, aiming to provide a deep-sea sediment sampling system and method based on image recognition technology. The system includes a main control unit, an image acquisition and recognition unit, an image storage unit, an illumination and camera unit, a sensor unit, a drive unit, an actuator, and a power supply unit. The main control unit is electrically connected to the image acquisition and recognition unit, the illumination and camera unit, the sensor unit, and the drive unit. The image acquisition and recognition unit is electrically connected to the illumination and camera unit and the image storage unit. The drive unit is connected to the actuator, and the power supply unit provides power to each module. This invention uses machine vision to control the sampler, enabling precise sampling operations. It can acquire sediment samples and maintain in-situ pressure, and includes an interface design for pushing samples to a pressure-holding transfer device. It also enables the long-sequence acquisition and storage of sensor data such as temperature and pressure, and integrates an active heat preservation function after sediment acquisition.

Description

Technical Field

[0001] This invention relates to deep-sea sampling technology, specifically to a deep-sea sediment sampling system and method based on image recognition technology. Background Technology

[0002] The deep sea generally refers to the ocean at a depth of over 1000 meters, accounting for 75% of the Earth's ocean volume. The deep sea is an extreme ecological environment characterized by high pressure, low temperature, and darkness. Microorganisms living in such extreme environments exhibit significantly different survival characteristics compared to terrestrial microorganisms. Areas below 6000 meters are called the abyss, where scientists have discovered signs of life. Life in such extreme environments is receiving increasing attention. Abyssal sediments contain a vast amount of microorganisms, and their immense diversity represents a huge treasure trove of natural biological products. Over the past fifty years, approximately 20,000 marine biological natural products have been discovered. Many of these possess anti-tumor, anti-cellular aging, antibacterial, and antiviral activities, and dozens of marine anticancer drugs have entered clinical or preclinical research stages. Currently, our understanding of life phenomena and processes in the abyss is very limited, and our exploration and understanding of abyssal organisms and genetic resources are insufficient. Studying reliable deep-sea sediment samples can provide a deeper understanding of the sedimentary environment throughout geological history, the engineering properties of sediments, and a better understanding of the changing marine environment. Furthermore, pressurized samples can provide living abyssal microorganisms needed for modern medical research.

[0003] Existing lander-based deep-sea samplers typically achieve deep-sea sampling through a combination of manual and mechanical assistance, with real-time image display. This places high demands on the operators' control skills. Furthermore, maintaining a high level of control is unsustainable for extended periods during high-precision sampling operations. In particular, sediment sampling devices suffer from poor integration and limited capabilities in areas such as deep depth, no need for mothership support, multi-signal acquisition, high-fidelity high-pressure processing, and integrated image acquisition and processing control. Therefore, it is necessary to develop a higher-performance deep-sea sampling system. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a deep-sea sediment sampling system and method based on image recognition technology, addressing the shortcomings of existing technologies.

[0005] To address the problems existing in the prior art, the present invention is achieved through the following technical solution:

[0006] A deep-sea sediment sampling system based on image recognition technology is provided, comprising a main control unit, an image acquisition and recognition unit, an image storage unit, an illumination and camera unit, a sensor unit, a drive unit, an actuator, and a power supply unit. The main control unit is electrically connected to the image acquisition and recognition unit, the illumination and camera unit, the sensor unit, and the drive unit. The image acquisition and recognition unit is electrically connected to the illumination and camera unit and the image storage unit. The drive unit is connected to the actuator. The power supply unit provides power to each unit module.

[0007] The main control unit is used to receive data transmitted from the sensor unit and the image acquisition and recognition unit, and to control the actuator to perform sampling through the drive unit;

[0008] The image acquisition and recognition unit is implemented using a programmable logic array (FPGA), and includes a digital image capture module, a storage module, a first frame buffer control module (before processing), a real-time image processing module, a second frame buffer control module (after processing), a debug interface module, a communication module, and an SCCB communication control module. The digital image capture module, the first frame buffer control module, the real-time image processing module, and the second frame buffer control module are electrically connected sequentially. The camera in the lighting and imaging unit is electrically connected to the digital image capture module and the SCCB communication control module, respectively. Electrical connections are also established between the digital image capture module and the storage module, between the first frame buffer control module and frame buffer SDRAM1 (before processing), between the second frame buffer control module and frame buffer SDRAM2 (after processing), and between the communication module and the main control unit.

[0009] The image acquisition and recognition unit obtains image information from the lighting and camera unit, processes and analyzes the real-time position information of the actuator, and feeds it back to the main control unit, which then controls the actions of the lighting and camera unit and the drive unit to manipulate the sampling behavior of the actuator.

[0010] This invention further provides a deep-sea sediment sampling method based on image recognition technology, including:

[0011] The lighting and camera unit acquires real-time position images of the sampler in the actuator and transmits the data to the image acquisition and recognition unit. After image processing, the image acquisition and recognition unit identifies the image to determine whether the actuator has reached the preset position. Then, the processed information is exchanged with the main control unit in real time, which controls the actions of the lighting and camera unit and the drive unit to manipulate the sampling behavior of the actuator.

[0012] The workflow of the image acquisition and recognition unit includes: after power-on, configuring the camera in the lighting and imaging unit using the SCCB communication control module; acquiring real-time digital image data streams through the digital image capture module, buffering them in the first frame buffer control module, and saving the data through the storage module; the real-time image processing module reading the image data streams in the first frame buffer control module and processing them, then storing them in the second frame buffer control module; the debugging interface module downloading the onboard program and configuring its functions; and the communication module communicating with the main control unit, which sends corresponding control commands to the drive unit based on the processing results of the real-time images to control the actions of the actuators.

[0013] When processing the image data stream, an underwater image restoration method is used to preprocess the data to obtain a clear underwater image. Then, the image is input into the previously trained model, and the pixel-distance conversion formula is used to calculate and locate the top position of the sampler in the image, so as to obtain the data of the sampler position and its extension distance in the actuator.

[0014] Compared with the prior art, the beneficial effects of the present invention are:

[0015] 1. This invention uses machine vision to control the sampler, which can accurately perform sampling operations.

[0016] 2. To achieve the acquisition of sediment samples and maintenance of in-situ pressure by mounting the sampling device on a 10,000-meter lander, and to implement the interface design for pushing samples to the pressure-maintaining transfer device, to realize the long-sequence acquisition and storage of sensor data such as temperature and pressure, and to integrate the active heat preservation function after sediment acquisition, through the acquisition and study of pressure-maintaining sediment samples, to elucidate the genomic structural characteristics of deep-sea multicellular organisms and the co-evolutionary mechanism between them and their symbiotic microorganisms, to establish an evaluation and development system for deep-sea biological genetic resources and metabolites, and to provide a foundation for subsequent development and utilization of biological resources and the establishment of life systems. Attached Figure Description

[0017] Figure 1 This is a system block diagram of the present invention;

[0018] Figure 2 This is a block diagram of the image acquisition and recognition unit of the present invention;

[0019] Figure 3 This is a schematic diagram of an example actuator;

[0020] Figure 4 This is a flowchart of the image recognition process of the present invention.

[0021] In the diagram: 1-lead screw; 2-sampling piston; 3-sampling connecting rod; 4-sampling cylinder; 5-sampling cone. Detailed Implementation

[0022] First, it should be noted that this invention relates to database technology, specifically an application of computer technology in the fields of data processing and control. The implementation of this invention involves the application of multiple software functional modules. The applicant believes that, after carefully reading the application documents and accurately understanding the implementation principles and objectives of this invention, and in conjunction with existing known technologies, those skilled in the art can fully utilize their software programming skills to implement this invention. All references made in this application fall within this scope, and the applicant will not list them all further.

[0023] Those skilled in the art will understand that, besides implementing a portion of the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0024] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] The following embodiments are intended to enable those skilled in the art to gain a more comprehensive understanding of the present invention, but do not limit the invention in any way.

[0026] like Figure 1As shown, a deep-sea sediment sampling system based on image recognition technology includes a main control unit, an image acquisition and recognition unit, an image storage unit, an illumination and camera unit, a sensor unit, a drive unit, an actuator, and a power supply unit. The main control unit is electrically connected to the image acquisition and recognition unit, the illumination and camera unit, the sensor unit, and the drive unit. The image acquisition and recognition unit is electrically connected to the illumination and camera unit and the image storage unit. The drive unit is connected to the actuator. The power supply unit provides power to each module.

[0027] The main control unit receives data from the sensor unit and image acquisition and recognition unit, and controls the actuator to perform sampling through the drive unit. The main control unit specifically comprises: a main control chip, an analog voltage / current signal acquisition module, a PT100 temperature acquisition module, a leakage detection module, a peripheral relay drive module, an RS232 communication module, an RS485 communication module, and a data storage module. The analog voltage / current signal acquisition module acquires the voltage signal from the external pressure sensor, converts it to digital signal, and then communicates with the main control chip via SPI communication. The PT100 temperature acquisition module... An external temperature sensor is connected to the acquisition device to feed back the corresponding voltage signal to the main control chip; a water leakage detection module is used to detect water leakage in the sampling system by connecting an external 100K resistor; an external relay drive module is used to control the power supply of the motor in the drive unit. When the motor is idle, the relay cuts off the motor power supply to reduce the power consumption of the entire system; an RS232 communication module is used to establish communication with the image acquisition and recognition unit and receive control commands from the image acquisition and recognition unit to drive the motor to perform corresponding actions; an RS485 communication module is used to communicate with the motor in the drive unit.

[0028] like Figure 2As shown, the image acquisition and recognition unit is implemented using a programmable logic array (FPGA), including a digital image capture module, a storage module, a first frame buffer control module (before processing), a real-time image processing module, a second frame buffer control module (after processing), a debug interface module, a communication module, and an SCCB communication control module. The digital image capture module, the first frame buffer control module, the real-time image processing module, and the second frame buffer control module are electrically connected sequentially. The camera in the lighting and camera unit is electrically connected to the digital image capture module and the SCCB communication control module, respectively. Electrical connections are also established between the digital image capture module and the storage module, between the first frame buffer control module and frame buffer SDRAM1 (before processing), between the second frame buffer control module and frame buffer SDRAM2 (after processing), and between the communication module and the main control unit. The image acquisition and recognition unit acquires image information from the lighting and camera unit, processes and analyzes the real-time position information of the actuator, and feeds it back to the main control unit, which then controls the actions of the lighting and camera unit and the drive unit, manipulating the sampling behavior of the actuator.

[0029] The drive unit includes an oil-filled motor consisting of a stepper motor and a motor housing; the enclosed motor housing is filled with oil to prevent the stepper motor from directly contacting the water, and the output shaft of the stepper motor is connected to the sampling piston in the actuator via a lead screw.

[0030] like Figure 3 As shown, as an example, the actuator is a cylindrical sampler with a sampling piston 2 built into its cavity. One end of the sampling piston 2 is connected to the drive unit via a lead screw 1, and the other end is equipped with a sampling connecting rod 3 arranged coaxially with the sampler. A sampling cone 5 is provided at the end of the sampling connecting rod 3, and the sampling cone 5 is equipped with a circumferential seal, which can close the opening end of the sampler when the sampling piston 2 retracts, realizing sample collection and pressure maintenance. When the sampler is filled with high-pressure water, the sampling cone 5 and the sampling piston 2 will not separate, achieving self-balancing at both ends. The image recognition unit identifies the position of the sampling cone 5, and the main control unit controls the drive unit based on the recognition result.

[0031] This invention further provides a deep-sea sediment sampling method based on image recognition technology, the process of which is as follows: Figure 4 As shown. The sampling method includes:

[0032] The lighting and camera unit acquires real-time position images of the sampler in the actuator and transmits the data to the image acquisition and recognition unit. After image processing, the image acquisition and recognition unit identifies the image to determine whether the actuator has reached the preset position. Then, the processed information is exchanged with the main control unit in real time, which controls the actions of the lighting and camera unit and the drive unit to manipulate the sampling behavior of the actuator.

[0033] The workflow of the image acquisition and recognition unit includes: after power-on, configuring the camera in the lighting and imaging unit using the SCCB communication control module; acquiring real-time digital image data streams through the digital image capture module, buffering them in the first frame buffer control module, and saving the data through the storage module; the real-time image processing module reading the image data streams in the first frame buffer control module and processing them, then storing them in the second frame buffer control module; the debugging interface module downloading the onboard program and configuring its functions; and the communication module communicating with the main control unit, which sends corresponding control commands to the drive unit based on the processing results of the real-time images to control the actions of the actuators.

[0034] When processing the image data stream, an underwater image restoration method is used to preprocess the data to obtain a clear underwater image. Then, the image is input into the previously trained model, and the pixel-distance conversion formula is used to calculate and locate the top position of the sampler in the image, so as to obtain the data of the sampler position and its extension distance in the actuator.

[0035] The underwater image restoration method includes:

[0036] The integral formula (1) is obtained by simplifying and optimizing the Jaffe-McGramery model:

[0037]

[0038] In the formula, E(f) represents the spatial form of the blurred image, E d (f) represents the spatial form of the direct component, representing the desired final image; f represents the gray value of a point (x, y) on a certain channel in the image;

[0039] The parameters b and k in the formula are calculated by a convolutional neural network trained on previously acquired deep-sea images. The convolutional neural network consists of a b-network module, a k-network module, and a J-estimator module. The b-network module estimates parameter b, the k-network module estimates parameter k, and the J-estimator module reconstructs the image. Both the b-network module and the k-network module include convolutional layers and max-pooling layers. The convolutional neural network uses three convolutional layers to extract features, two max-pooling layers to overcome local sensitivity and reduce the resolution of the feature map, and the last layer is a convolutional layer for nonlinear regression. A ReLU layer is added after each convolutional layer to avoid slow convergence and local minima during training.

[0040] The convolutional neural network uses low-resolution underwater images generated by depth maps and underwater optical models as training samples for the network. When processing the image data stream, the input is a real-time underwater sampled image, and the output is that parameters k and b are real values ​​respectively. After calculating the values ​​of the two parameters through the b network module and the k network module respectively, the underwater image is restored through formula (1) in the J estimator module, and finally the RGB three-channel underwater real image is obtained.

[0041] The calculation and localization of the top position of the detection sampler in the image specifically includes:

[0042] The bounding box proposed by the mask R-CNN model is defined as the Region of Interest (ROI). Morphological opening operations are performed on the entire ROI to eliminate small particles. The region within the ROI is then binarized. Since the deep sea is a dark environment and the device's light source is the only active light source, the RGB pixel values ​​of the sampler and the background device remain basically consistent under the same light source. Based on the effective classification thresholds of the sampler and the background device obtained from previous experiments, binarization is performed. Target pixels near the bottom edge within the ROI are selected, and a cone-shaped determination is applied to these pixels. That is, with the pixel as the center, the two straight lines passing through the pixel with the largest angle must have an angle between 0° and 60°. If the condition is met, the pixel is considered the top of the sampler, and the base pixel is the target pixel closest to the top within the ROI. The scaling distance of the sampler is calculated in real time using the pixel-distance conversion formula of the relative position between the camera device and the sampler, which was measured in advance.

[0043] The training methods for the aforementioned pre-trained model include:

[0044] First, underwater image samples containing the sampler are collected, and then manually labeled before being used to train the mask-rcnn model;

[0045] For the sampler, the model loss function is improved as follows:

[0046]

[0047] In the formula, (x, y, w, h) represents the coordinates of the top-left corner vertex of the prediction box and the width and height of the border .... a y a w a h a (x) represents the coordinates of the top-left corner of the candidate box and the width and height of the border. * y * w * h * (t) represents the coordinates of the top-left corner vertex of the target bounding box, along with the width and height of the bounding box; x , t y , tw , t h (t) represents the affine transformation information between the predicted bounding box and the candidate bounding box. x * , t y * , t w * , t h * ) represents the affine transformation information between the target box and the candidate box.

[0048] To enable the training process to accept negative sample data, a conditional judgment is added to the training process of the predicted bounding box during RPN: when the four localization information of the target (x, y, z) is received, a conditional judgment is added: when the target's four localization information ... * y * w * h * When both ) are 0, the affine transformation information (t) between the predicted box and the candidate box x , t y , t w , t h All values ​​are set to 0, and the affine transformation information (t) between the target box and the candidate box is set to 0. x * , t y * , t w * , t h * The value is also set to 0; this ensures that no division by zero error will occur during the functional programming implementation, and that the training can run normally.

[0049] Finally, it should be noted that the above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of this invention should be considered within the scope of protection of this invention.

Claims

1. A method for deep-sea sediment sampling based on image recognition technology, characterized in that, This method is based on a deep-sea sediment sampling system. This system includes a main control unit, an image acquisition and recognition unit, an image storage unit, an illumination and camera unit, a sensor unit, a drive unit, an actuator, and a power supply unit. The main control unit is electrically connected to the image acquisition and recognition unit, the illumination and camera unit, the sensor unit, and the drive unit. The image acquisition and recognition unit is electrically connected to the illumination and camera unit and the image storage unit. The drive unit is connected to the actuator, and the power supply unit provides power to each module. The main control unit is used to receive data transmitted from the sensor unit and the image acquisition and recognition unit, and to control the actuator to perform sampling through the drive unit; The image acquisition and recognition unit is implemented using a programmable logic array (FPGA), and includes a digital image capture module, a storage module, a first frame buffer control module, a real-time image processing module, a second frame buffer control module, a debug interface module, a communication module, and an SCCB communication control module. The digital image capture module, the first frame buffer control module, the real-time image processing module, and the second frame buffer control module are electrically connected sequentially. The camera in the lighting and imaging unit is electrically connected to the digital image capture module and the SCCB communication control module, respectively. Electrical connections are also established between the digital image capture module and the storage module, between the first frame buffer control module and frame buffer SDRAM1, between the second frame buffer control module and frame buffer SDRAM2, and between the communication module and the main control unit. The image acquisition and recognition unit acquires image information from the lighting and camera unit, processes and analyzes the real-time position information of the actuator, and feeds it back to the main control unit, which then controls the actions of the lighting and camera unit and the drive unit to manipulate the sampling behavior of the actuator. This deep-sea sediment sampling method based on image recognition technology includes: The lighting and camera unit acquires real-time position images of the sampler in the actuator and transmits the data to the image acquisition and recognition unit. After image processing, the image acquisition and recognition unit identifies the image to determine whether the actuator has reached the preset position. Then, the processed information is exchanged with the main control unit in real time, which controls the actions of the lighting and camera unit and the drive unit to manipulate the sampling behavior of the actuator. The workflow of the image acquisition and recognition unit includes: after power-on, configuring the camera in the lighting and imaging unit using the SCCB communication control module; acquiring real-time digital image data streams through the digital image capture module, buffering them in the first frame buffer control module, and saving the data through the storage module; the real-time image processing module reading the image data streams in the first frame buffer control module and processing them, then storing them in the second frame buffer control module; the debugging interface module downloading the onboard program and configuring its functions; and the communication module communicating with the main control unit, which sends corresponding control commands to the drive unit based on the processing results of the real-time images to control the actions of the actuators. When processing the image data stream, an underwater image restoration method is used to preprocess the data to obtain a clear underwater image. Then, the image is input into the previously trained model, and the pixel-distance conversion formula is used to calculate and locate the top position of the sampler in the image to obtain the data of the sampler position and its extension distance in the actuator. The underwater image restoration method includes: Formula (1) is obtained by simplifying and optimizing the Jaffe-McGramery model: In the formula, E(f) represents the spatial form of the blurred image, E d (f) represents the spatial form of the direct component, which represents the desired final image; f represents the gray value of a point (x,y) on a certain channel in the image; Among them, parameters k and b are calculated by a convolutional neural network trained with images collected in the deep sea in the early stage; This convolutional neural network consists of a b network module, a k network module, and a J estimator module. The b network module estimates parameter b, the k network module estimates parameter k, and the J estimator module reconstructs the image. Both the b and k network modules include convolutional layers and max-pooling layers. The convolutional neural network uses three convolutional layers to extract features, two max-pooling layers to overcome local sensitivity and reduce the resolution of the feature map, and a final convolutional layer for non-linear regression. A ReLU layer is added after each convolutional layer to avoid slow convergence and local minima during training. The convolutional neural network uses low-resolution underwater images generated by depth maps and underwater optical models as training samples for the network. When processing the image data stream, the input is a real-time underwater sampled image, and the output is parameters k and b. After calculating the values ​​of the two parameters through the b network module and the k network module respectively, the underwater image is restored through formula (1) in the J estimator module, and finally the real underwater image of RGB three channels is obtained.

2. The method according to claim 1, characterized in that, The main control unit comprises: a main control chip, an analog voltage / current signal acquisition module, a PT100 temperature acquisition module, a leakage detection module, a peripheral relay drive module, an RS232 communication module, an RS485 communication module, and a data storage module; wherein... The analog voltage / current signal acquisition module is used to acquire the voltage signal from the external pressure sensor of the device, and after analog-to-digital conversion, it interacts with the main control chip via SPI communication. The PT100 temperature acquisition module is used to acquire the temperature signals from external temperature sensors connected to the device and feed the corresponding voltage signals back to the main control chip. The water leakage detection module is used to detect water leakage in the equipment of the sampling system by using an external 100K resistor. The peripheral relay drive module is used to control the power supply of the motor in the drive unit. When the motor is in an idle state, the relay cuts off the power supply to the motor to reduce the power consumption of the entire system. The RS232 communication module is used to establish a connection with the image acquisition and recognition unit and receive control commands from the image acquisition and recognition unit to drive the motor to perform corresponding actions. The RS485 communication module is used to communicate with the motor in the drive unit.

3. The method according to claim 1, characterized in that, The drive unit includes an oil-filled motor consisting of a stepper motor and a motor housing; the enclosed motor housing is filled with oil to prevent the stepper motor from directly contacting the water, and the output shaft of the stepper motor is connected to the sampling piston in the actuator via a lead screw.

4. The method according to claim 1, characterized in that, The actuator is a cylindrical sampler with a sampling piston built into its cavity. One end of the sampling piston is connected to the drive unit, and the other end is provided with a sampling connecting rod arranged coaxially with the sampler. A sampling cone is provided at the end of the sampling connecting rod, and a circumferential seal is provided on the sampling cone, which can close the opening end of the sampler when the sampling piston is retracted, so as to realize the collection and pressure holding of the sample.

5. The method according to claim 1, characterized in that, The calculation and localization of the top position of the detection sampler in the image specifically includes: The bounding box proposed by the mask R-CNN model is defined as the Region of Interest (ROI). Morphological opening operations are performed on the entire ROI to eliminate small particles. The region within the ROI is then binarized. Since the deep sea is a dark environment and the device's light source is the only active light source, the RGB pixel values ​​of the sampler and the background device remain basically consistent under the same light source. Based on the effective classification thresholds of the sampler and the background device obtained from previous experiments, binarization is performed. Target pixels near the bottom edge within the ROI are selected, and a cone-shaped determination is applied to these pixels. That is, with the pixel as the center, the two straight lines passing through the pixel with the largest angle must have an angle between 0° and 60°. If the condition is met, the pixel is considered the top of the sampler, and the base pixel is the target pixel closest to the top within the ROI. The scaling distance of the sampler is calculated in real time using the pixel-distance conversion formula of the relative position between the camera device and the sampler, which was measured in advance.

6. The method according to claim 1, characterized in that, The training methods for the aforementioned pre-trained model include: First, underwater image samples containing the sampler are collected, and then manually labeled before being used to train the mask-rcnn model; For the sampler, the model loss function is improved as follows: In the formula, (x, y, w, h) represents the coordinates of the top-left corner vertex of the prediction box and the width and height of the border .... a y a w a h a (x*, y*, w*, h*) represents the coordinates of the top-left corner of the candidate bounding box and the width and height of its border; (t) represents the coordinates of the top-left corner of the target bounding box and the width and height of its border. x , t y , t w , t h (t) represents the affine transformation information between the predicted bounding box and the candidate bounding box. x * , t y * , t w * , t h * () represents the affine transformation information between the target bounding box and the candidate bounding box; To enable the training process to accept negative sample data, a conditional judgment is added to the training process of the predicted bounding boxes during RPN: when all four localization information (x*, y*, w*, h*) of the target are 0, the affine transformation information (t) between the predicted bounding box and the candidate bounding box is changed. x , t y , t w , t h All values ​​are set to 0, and the affine transformation information (t) between the target box and the candidate box is set to 0. x *, t y *, t w *, t h *) is also set to 0; this ensures that no division by zero error will occur during the functional programming implementation, and that training can run normally.

Images