Digital display method and system for water search and rescue exercise

Abstract

The invention provides a digital display method and system for water search and rescue exercises, and the method comprises the steps: carrying out the instance segmentation of an exercise video, thereby recognizing a participating unit in the video, and positioning the position of the participating unit in each frame of video; the associated participating units and the corresponding exercise information form operable links, and the exercise information is embedded into the corresponding links and stored in a database; digitally displaying the processed video and corresponding information; interactive operation is provided, operation input is responded, and a result is returned, so that the functions of segmenting a target instance, fusing multi-target information and interactively and digitally displaying exercise information are realized, and the problems of limited viewing angle, monotonous information transmission, passive exercise information acquisition and lack of interactivity in viewing and emulation are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of video processing, and particularly relates to a digital display method and system for water search and rescue exercises. Background Art

[0002] Currently, for the viewing methods of water search and rescue exercises, the existing technologies mainly rely on two forms: on-site viewing and video live streaming. However, due to the vastness of the exercise scene, the diversity of exercise units, and the continuity of exercise time, these viewing methods have obvious limitations, mainly including: the perspective of on-site audiences is limited, mainly relying on the perspective of the viewing platform or live streaming equipment; the information acquisition method is passive, mainly understanding the exercise situation through on-site commentary; there is a lack of interactivity, which easily leads to boredom among audiences. With the development of network technology and the popularization of smart devices, digital viewing methods can be obtained through smart devices such as mobile phones and tablets, thereby obtaining richer information.

[0003] In the field of computer vision, instance segmentation combines the characteristics of object detection and semantic segmentation, and can accurately identify and label different individuals within the same category. This technology has important application value in water search and rescue exercises, meeting the need to identify and label exercise units. Further, attach corresponding exercise information to the participating units in the video frame, and can update dynamic information as the exercise progresses. For this reason, the present invention proposes a digital display system for water search and rescue exercises, which designs and constructs a visualization platform based on a web page to achieve an interactive display effect. This system can realize functions such as target instance segmentation, multi-target information fusion, and interactive digital display, and has important application value and practical significance.

[0004] At the present stage, the viewing methods of water search and rescue exercises have not deeply integrated digital display methods. The viewing platform has a limited perspective and it is difficult to view the overall exercise scene. Live streaming through drones can provide perspectives outside the viewing platform, but there is a risk of affecting participating units, and the perspective is still limited, making it difficult to obtain rich information. The on-site commentary conveys information monotonously, with insufficient information capacity, the information acquisition method is passive, and there is a lack of real-time interaction. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to provide a digital display method and system for water search and rescue exercises, which are used for segmenting target instances, fusing multi-target information, and interactively digitally displaying exercise information.

[0006] The technical solution adopted by the present invention to solve the above technical problem is: a digital display method for water search and rescue exercises, comprising the following steps: S0: Obtain an exercise video including participating units; S1: Build and train a fully convolutional network model; perform instance segmentation on the targets in the exercise video to identify participating units; S2: Associate the identified participating units with the corresponding exercise information to form an operable link, embed the exercise information in the corresponding link, and store it in the database; S3: Digitally display the processed video and the corresponding fusion information; S4: Provide an interactive operation, respond to the operation input, and return the result.

[0007] According to the above solution, in step S1, the specific steps are as follows: S11: Build a fully convolutional network model; S12: Input the exercise video including the participating units into the output layer of the fully convolutional network model for training to obtain the parameters and model that can identify the exercise units; S13: Perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the positions of the participating units in each frame of the video.

[0008] Furthermore, in step S11, the fully convolutional network model uses Resnet18 as the main body, retains the structure and parameters of the convolutional layer of the original model, and replaces the last global average pooling layer and fully connected layer with a 1×1 convolutional layer and a transposed convolutional layer; the fully convolutional network model includes an input layer, a series of residual blocks, a convolutional layer, and an output layer.

[0009] Furthermore, in step S12, the specific steps are as follows: S121: When inputting each frame image of the video into the input layer and passing through the series of residual blocks in sequence, the number of feature channels extracted becomes larger and the feature size is halved layer by layer; S122: When passing through the convolutional layer, use a 1×1 convolutional layer to transform the number of feature channels into the number of instances; S123: Use the output layer to transform the height and width into the size of the input image; S124: The model outputs a class prediction with the same size as the input image and the number of channels including the pixel positions of the image; obtain the mask point set of each participating unit in each frame image of the video through the color mapping function.

[0010] Furthermore, in step S121, for the series of residual blocks, the initial parameters use the pre-trained parameters of Resnet18 and are fine-tuned with a smaller learning rate during training; In step S122, for the 1×1 convolutional layer, use the Xavier method to initialize the parameters so that the gradient variance remains unchanged during the forward and backward propagation processes.

[0011] Further, in the step S123, the parameters of the output layer are initialized using the bilinear interpolation method, so that the weight of the center point of the convolution kernel of the output layer is the largest, and the edge weight decreases as the distance from the center increases; the padding and stride of the output layer are determined by the input size, convolution kernel size, padding, and stride in the vertical and horizontal directions.

[0012] Further, in the step S13, the specific steps are as follows: S131: Determine the number of instances according to the type and quantity of participating units; S132: Create a training label for instance segmentation, and establish a label matrix with the same size as the corresponding training image; assign a unique RGB value to the label of each participating unit, and establish a color mapping table; S133: Represent the RGB color value of each pixel in the matrix, that is, the pixel color of the label, through the color mapping table and the background pixel representation of non-participating unit instances; S134: Map the label matrix to an RGB image through a color mapping function.

[0013] According to the above solution, in the step S2, the specific steps are as follows: S21: Associate the mask point set of the participating units in each frame of the video with the exercise information; S22: Correlate the id of the participating unit with the participating unit through the exercise information, and form a link between the exercise information and the corresponding participating unit; S23: Store the display information of each frame of the video in the database in the format of video frame time - participating unit id - mask point set - exercise information.

[0014] A digital display system for water search and rescue exercises A video acquisition sub-module, used to acquire an exercise video including participating units; An image recognition sub-module, used to perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the position of the participating units in each frame of the video; A video processing and storage sub-module, used to associate the mask point set of the identified participating units with the corresponding exercise information, form an operable link, embed the exercise information into the corresponding link, and store it in the database; A digital display sub-module, used to digitally display the processed video and the corresponding information; An operation response sub-module, used to provide interactive operations, respond to operation inputs, and return results.

[0015] A computer memory, which stores a computer program executable by a computer processor, and the computer program executes a digital display method for water search and rescue exercises.

[0016] The beneficial effects of the present invention are as follows: 1. A digital display method and system for water search and rescue drills according to the present invention identify participating units in a drill video by instance segmentation of the video, and locate the positions of the participating units in each frame of the video; associate the participating units with corresponding drill information to form an operable link, embed the drill information into the corresponding link and store it in a database; digitally display the processed video and corresponding information; provide interactive operations, respond to operation inputs and return results, realizing the functions of segmenting target instances, fusing multi-target information, and interactively digitally displaying drill information.

[0017] 2. The present invention builds a drill scenario based on a navigation simulator to highly restore the drill scenario and sea conditions, and makes a drill video by arranging virtual cameras from multiple angles, solving the problem of limited viewing angles.

[0018] 3. The present invention obtains the mask point set of participating units based on instance segmentation technology, associates drill information with video frames through multi-target information fusion, realizes dynamic update of the information of each participating unit, and solves the problem of monotonous information transmission.

[0019] 4. The present invention builds a web page based on a server and a database to implement a digital interactive display platform, digitally displays multi-target fusion information, realizes active acquisition of required drill information, and solves the problems of passive acquisition of drill information and lack of interactivity in observation.

[0020] Of course, it is not necessary for any product implementing the present invention to simultaneously achieve all the above-mentioned advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is the structural diagram of the embodiment of the present invention.

[0023] Figure 2 It is the flowchart of the embodiment of the present invention.

[0024] Figure 3 It is the full convolutional network framework diagram of the embodiment of the present invention.

[0025] Figure 4 It is the distribution diagram of the classification and instance quantity of participating units in the embodiment of the present invention.

[0026] Figure 5 It is the multi - target information fusion flow chart of the embodiment of the present invention.

[0027] Figure 6 It is the schematic diagram of the layout of the digital display interface of the embodiment of the present invention. Specific Embodiments

[0028] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] Embodiment 1 See Figure 1 , the specific steps of a digital display method for a water search and rescue exercise are as follows: S0: Obtain the exercise video including participating units; S1: Build and train a fully convolutional network model; perform instance segmentation on the targets in the exercise video to identify the participating units; S2: Associate the identified participating units with the corresponding exercise information, form an operable link, embed the exercise information into the corresponding link, and store it in the database; S3: Digitally display the processed video and the corresponding fusion information; S4: Provide an interactive operation, respond to the operation input, and return the result.

[0030] Further, in step S1, the specific steps are: S11: Build a fully convolutional network model; S12: Input the exercise video including participating units into the output layer of the fully convolutional network model training to obtain the parameters and model for identifying the exercise units; S13: Perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the positions of the participating units in each frame of the video.

[0031] Further, in step S11, the fully convolutional network model uses Resnet18 as the main body, retains the structure and parameters of the convolutional layer of the original model, and replaces the last global average pooling layer and fully connected layer with a 1×1 convolutional layer and a transposed convolutional layer; the fully convolutional network model includes an input layer, a series of residual blocks, a convolutional layer, and an output layer.

[0032] Further, in step S12, the specific steps are: S121: When inputting each frame image of the video into the input layer and passing through the series of residual blocks in turn, the number of feature channels extracted becomes larger and the feature size is halved layer by layer; S122: When passing through the convolutional layer, the number of feature channels is transformed into the number of instances through a 1×1 convolutional layer; S123: The height and width are transformed into the size of the input image through the output layer; S124: The model outputs a class prediction where the output size is the same as the input image size and the number of channels contains the pixels of the image positions; the mask point sets of each participating unit in each frame of the video are obtained through the color mapping function.

[0033] Furthermore, in step S121, for the cascaded residual blocks, the initial parameters adopt the pre-trained parameters of Resnet18 and are fine-tuned with a relatively small learning rate during training; In the described step S122, for the 1×1 convolutional layer, the parameters are initialized using the Xavier method so that the gradient variance remains unchanged during the forward and backward propagation processes.

[0034] Furthermore, in step S123, for the output layer, the parameters are initialized using the bilinear interpolation method so that the weight of the center point of the convolutional kernel of the output layer is the largest and the weight at the edge decreases as the distance from the center increases; the padding and stride of the output layer are determined by the input size, convolutional kernel size, padding, and stride in the vertical and horizontal directions.

[0035] In step S13, the specific steps are as follows: S131: Determine the number of instances according to the types and quantities of the participating units; S132: Create the training labels for instance segmentation and establish a label matrix of the same size as the corresponding training image; assign a unique RGB value to the label of each participating unit and establish a color mapping table; S133: Represent the RGB color value of each pixel in the matrix, that is, the pixel color of the label, through the color mapping table and the background pixel representation of non-participating unit instances; S134: Map the label matrix to an RGB image through the color mapping function.

[0036] In step S2, the specific steps are as follows: S21: Associate the mask point sets of the participating units in each frame of the video with the exercise information; S22: Correlate the id of the participating unit with the participating unit through the exercise information and form a link between the exercise information and the corresponding participating unit; S23: Store the display information of each frame of the video in the database in the format of video frame time - participating unit id - mask point set - exercise information.

[0037] In this embodiment, the exercise video is subjected to instance segmentation to identify the participating units in the video, and the positions of the participating units in each frame of the video are located; the participating units are associated with the corresponding exercise information to form an operable link, the exercise information is embedded in the corresponding link and stored in the database; the processed video and the corresponding information are digitally displayed; and interactive operations are provided, the operation inputs are responded to and results are returned, realizing the functions of segmenting target instances, integrating multi-target information, and interactively digitally displaying exercise information.

[0038] Embodiment 2 The steps of this embodiment are the same as those of Embodiment 1, except that each step is applied to a specific instance. Specifically, the following steps are included: S0: Obtain an exercise video including participating units; The source of the exercise video is the Navi-Trainer Professional 5000 marine simulator. By building the corresponding exercise scenario in the simulator according to the pre-arranged exercise script and recording with the virtual camera built in the marine simulator during the simulation run.

[0039] S1: Build and train a fully convolutional network model; perform instance segmentation on the targets in the exercise video to identify the participating units. The specific steps are as follows: S11: Build a fully convolutional network model; The fully convolutional network model uses Resnet18 as the main body, retains the structure and parameters of the convolutional layer of the original model, and replaces the last global average pooling layer and fully connected layer with a 1×1 convolutional layer and a transposed convolutional layer; the main framework net of the network includes an input layer conv1, 4 cascaded residual blocks blk1, blk2, blk3, blk4, a convolutional layer conv2, and an output layer tconv.

[0040] S12: Input the exercise video including participating units into the training output layer of the fully convolutional network model to obtain the parameters and model for recognizing the exercise units. The specific steps are as follows: S121: Input each frame image of the video into conv1, and then pass through blk1, blk2, blk3, blk4 in sequence. In this process, the number of extracted feature channels gradually increases, being 64, 64, 128, 256, 512 respectively, while the feature size is halved layer by layer, and finally reduced to 1 / 32 of the original image; S122: When passing through the convolutional layer conv2, the number of feature channels is transformed into the number of instances through a 1×1 convolutional kernel ; S123: Finally, the height and width are transformed into the size of the input image through the transposed convolutional layer tconv. The padding and stride of tconv follow the following formula:

[0041] Among them, and represent the vertical and horizontal directions respectively, represents the input size, represents the convolutional kernel size, represents the padding, represents the stride.

[0042] S124: The model output size is the same as the input image size The number of channels contains the class prediction of the pixels at the image positions. The prediction value and the label conversion process are as follows:

[0043]

[0044] Among them, represents the model output value, is the predicted class of the th pixel. Through the color mapping function, the mask point set of each participating unit in each frame of the video is obtained.

[0045] For conv1, blk1, blk2, blk3, blk4, the initial parameters adopt the pre-trained parameters of Resnet18 and are fine-tuned with a smaller learning rate during training.

[0046] For the 1×1 convolutional layer conv2, the Xavier method is used to initialize the parameters so that the gradient variance remains unchanged during the forward and backward propagation processes. The initialization formula is as follows:

[0047] Among them, represents the uniform distribution. The weight at the th position of the convolutional kernel is drawn from the uniform distribution. 512 is the number of input feature channels, is the number of output feature channels.

[0048] For the transposed convolutional layer tconv, the bilinear interpolation method is used to initialize the parameters so that the convolutional kernel of the transposed convolutional layer has the characteristics that the weight at the center point is the largest and the weight at the edge decreases with the increase of the distance from the center. The initialization formula is as follows:

[0049] Among them, represents the weight at the th position of the convolutional kernel, , represent the height and width of the convolutional kernel respectively.

[0050] S13: Perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the positions of the participating units in each frame of the video. The specific steps are as follows: S131: Let be the type of the participating unit, be the number of the -th type of participating unit. Determine the number of instances according to the type and number of the participating units : ; S132: Create the training labels for instance segmentation. The label of each training image with a size of is composed of a label matrix of the same size. Each element in the matrix represents a type of label, and the value range is ; 0 represents a non-participating unit instance, which is uniformly the background. are the participating units. Let each represent the RGB color value label of the -th participating unit, and the value is unique and not black [0,0,0]. Define the color mapping table as: ; S133: Let represent the RGB color value of the -th pixel in the matrix. The background pixels of non-participating unit instances are black [0,0,0]. The pixel color of the label is: ; S134: Map the label matrix to the RGB image . To simplify the label conversion process, define the color mapping function . The RGB color value corresponding to the label is: .

[0051] S2: Associate the mask point set of the identified participating units with the corresponding exercise information to form an operable link, embed the exercise information into the corresponding link, and store it in the database. The specific steps are as follows: S21: Associate the mask point set of the participating units in each frame of the video with the exercise information; Import the exercise information through the data input interface, and the content format is the participating unit id - start and end times of dynamic information - static information - dynamic information; the static information includes the name, type, and size of the participating unit; the dynamic information includes the status of the participating unit, the current task, and the movement parameters; the static information does not change, and the dynamic information is updated as the exercise progresses; For the exercise information that needs to be displayed in a special format, it is based on HTML tags; the format marking rules applicable to the exercise information are:

[0052] S22: Correlate the id of the participating unit with the participating unit through the exercise information, and form a link between the exercise information and the corresponding participating unit; S23: Store the display information of each frame of the video in the database in the format of video frame time - participating unit id - mask point set - exercise information.

[0053] S3: Digitally display the processed video and the corresponding fusion information; Use a web page as the display platform, including a side navigation bar, a video playback component, and an information display window.

[0054] S4: Provide interactive operations, respond to operation inputs, and return the results to the web page.

[0055] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0056] Embodiment 3 Such as Figure 1 and Figure 2 As shown, this embodiment is used to implement the principle of the above method embodiment to construct a digital display system for water search and rescue exercises, including a video acquisition sub-module, an image recognition sub-module, a video processing and storage sub-module, a digital display sub-module, and an operation response sub-module.

[0057] The video acquisition sub-module is used to acquire the exercise video including the participating units; The exercise video is recorded by the built-in virtual camera of the navigation simulator by building the corresponding exercise scene according to the pre-arranged exercise script in the simulator. The built-in function module of the simulator builds the exercise scene, and the "Environment settings" module adjusts environmental parameters such as time, weather, wind, waves, current, visibility, and fog, and adjusts the working conditions to conform to the exercise scenario; the "New Object" module adds objects, including exercise ships, personnel, equipment, action routes, and virtual cameras, and creates corresponding instances, such as Figure 4As shown, after presetting the speeds and directions of each point on the action route, the exercise objects bound to the route will act according to the route. The virtual camera has two modes: fixed free view and following target view. In the fixed free view mode, the camera is fixed at a certain point, and its view can be manually adjusted during the simulation process; in the following target view mode, the camera always follows the specified target, and the relative position between the camera and the target is adjusted by setting the X, Y, and Z parameters. Select the fixed free view or following target view according to the script, and select the appropriate view and position to record the exercise scenario.

[0058] The image recognition sub-module is used to perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the positions of the participating units in each frame of the video; The functions of the image recognition sub-module include training the model and instance segmentation. Training the model enables the image recognition module to identify the participating units and perform instance segmentation. The fully convolutional network is used for training, the cross-entropy loss method is used for the loss function, and the stochastic gradient descent method is used for the optimization function. The parameters and model obtained after training are saved as a.pth file for instance segmentation of the exercise video. During instance segmentation, the exercise video is imported through the data input interface, and the image recognition module calls the.pth file for instance segmentation. The recognition result format is: video frame time - participating unit id - mask point set, where the video frame time is stored in the format of hours: minutes: seconds, milliseconds (00:00:00,000).

[0059]

[0060] The above-mentioned fully convolutional network is as Figure 3 shown. The framework is improved based on the Resnet pre-trained model. Resnet18 is used as the main body, the convolutional layers of the source model are retained, and the last global average pooling layer and fully connected layer are replaced with 1×1 convolutional layer and transposed convolutional layer to form a fully convolutional network. All the framework designs and parameters on the Resnet source model are retained, and the output layer will be trained from scratch, and the parameters of the source model will be fine-tuned on the original basis.

[0061]

[0062] The video processing and storage sub-module is used to associate the mask point set of the identified participating units with the corresponding exercise information to form an operable link, embed the exercise information into the corresponding link, and store it in the database; The functions of the video processing and storage sub-module include multi-object information fusion and data storage. Multi-object information fusion associates the participating units in the video frame with the exercise information. The mask point set of the participating units comes from the image recognition module, and the exercise information comes from the data input interface. The two are associated through the participating unit ID. Among them, the dynamic information is additionally associated through time. After associating the target and time respectively, multi-object information fusion is achieved. Data storage stores the fused information in the database. The data format is: video frame time - participating unit ID - mask point set - static information - dynamic information, where the video frame time is stored in the format of hour:minute:second, millisecond (00:00:00,000), such as Figure 5 shown.

[0063]

[0064] The above exercise information is saved as a.txt file. The content format is: participating unit ID - dynamic information effective time - static information - dynamic information, where the start and end times of the dynamic information are stored in the format of hour:minute:second, millisecond (00:00:00,000). The static information includes the name, type, and size of the participating unit, and the dynamic information includes the status, current task, and motion parameters of the participating unit.

[0065] The digital display sub-module is used for digital display of the processed video and corresponding information; The digital display sub-module digitally displays the data in the database on the web page through a deployed server, mainly consisting of a side navigation bar, a video playback component, and an information display window, as Figure 6 shown. The side navigation bar lists the names of all videos, and there is a real-time exercise progress bar on the left to show the current exercise progress. After clicking on the corresponding name, the information display window will load the exercise video introduction, and the video playback component will load the corresponding exercise video and exercise information. The video playback component plays the selected exercise video, and clicking on the participating unit in the video can display the corresponding exercise information. The information display window will display the overall introduction of the exercise when first entering the web page, and will display the introduction of the corresponding exercise video when an exercise video is selected.

[0066] The operation response sub-module is used to provide interactive operations, respond to operation inputs, and return the results to the web page.

[0067] The functions of the operation response sub-module include communicating with the server periodically and updating web page information, obtaining input operations, transmitting operation information to the server and obtaining response results, and triggering different behaviors according to different response functions. The getProgress function will periodically obtain the real-time exercise progress and update the progress bar, obtain the current exercise progress from the server and compare it with the local time. When it is recognized that the current time is not within the exercise time or after the exercise ends, the progress bar will be hidden. After selecting a video in the side navigation bar, the showInfo function is triggered to grab the corresponding video data stream from the server and the exercise information in the database, and display the corresponding video introduction in the information display window. Subsequently, the bindMessage function and the applyStyle function are triggered. The bindMessage function associates the exercise information in the database with the video frame by frame and automatically updates the HTML code. The applyStyle function will recognize the special format tags in the exercise information and convert them into HTML code, so as to display the special format on the web page. Interactive operations are provided during video playback. After clicking on the location of the participating unit in the video, the clickObject function is triggered, and the corresponding participating information is extracted according to the participating unit id bound by the mask, and the HTML code is updated and the information is displayed.

[0068] It should be noted that according to the needs of implementation, each step / component described in this application can be split into more steps / components, or two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0069] This embodiment also includes a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; a computer program is stored in the memory, and when the program is executed by the processor, the processor is caused to execute the steps of a digital display method for a water search and rescue exercise.

[0070] This embodiment also provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by the processor, the processor is caused to implement a digital display method for a water search and rescue exercise.

[0071] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0072] Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0073] This application is described with reference to the flowcharts of the method and computer program product according to Embodiment 1 of this application and the block diagrams of the device (system) according to Embodiment 3. It should be understood that each process or block in the flowchart or block diagram can be implemented by computer program instructions, as well as the combination of processes or blocks in the flowchart or block diagram.

[0074] These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a digital display system for water search and rescue exercises for implementing the functions specified in one process Figure 1 one process or multiple processes or blocks Figure 1 or multiple blocks.

[0075] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 one process or multiple processes or blocks Figure 1 or multiple blocks.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide the steps of a digital display method for water search and rescue exercises for implementing the functions specified in one process Figure 1 one process or multiple processes or blocks Figure 1 or multiple blocks.

[0077] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made according to the principles and design concepts disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A digital display method for water search and rescue exercises, characterized in that: The following steps are involved: S0: Obtain exercise video including participating units; S1: Build and train a fully convolutional network model; perform instance segmentation on objects in exercise videos and identify participating units; S2: Associating the identified participating units with the corresponding exercise information to form an operational link, embedding the exercise information into the corresponding link, and storing it in the database; S3: Digital display of processed video and corresponding fusion information; S4: Provides interactive operations, responds to operation inputs, and returns results.

2. A digital display method for water search and rescue exercises according to claim 1, characterized in that: In the step S1, the specific steps are: S11: Build a fully convolutional network model; S12: Inputting the exercise video including the participating units into the training output layer of the full convolutional network model to obtain parameters and models that can identify the exercise units; S13: Perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the position of the participating units in each frame of the video.

3. A digital display method for water search and rescue exercises according to claim 2, characterized in that: In the step S11, the full convolutional network model uses Resnet18 as the main body, retains the structure and parameters of the convolutional layer of the source model, and replaces the last global average pooling layer and the fully connected layer with a 1×1 convolutional layer and a transposed convolutional layer; the full convolutional network model includes an input layer, a series of residual blocks, a convolutional layer and an output layer.

4. A digital display method for water search and rescue exercises according to claim 3, characterized in that: In the step S12, the specific steps are: S121: When each frame of the video is input into the input layer and passes through the series of residual blocks, the number of extracted feature channels increases and the feature size is halved layer by layer; S122: When passing through the convolution layer, the number of feature channels is converted into the number of instances through a 1×1 convolution layer; S123: transform the height and width to the size of the input image through the output layer; S124: The model output size is the same as the input image size, and the number of channels includes the category prediction of the pixels at the image position; the mask point set of each participating unit in each frame image of the video is obtained through the color mapping function.

5. A digital display method for water search and rescue exercises according to claim 4, characterized in that: In the step S121, for the series-connected residual blocks, the initial parameters use the Resnet18 pre-training parameters, and a smaller learning rate is used for fine-tuning during training; In step S122, for the 1×1 convolutional layer, the Xavier method is used to initialize parameters so that the gradient variance remains unchanged during forward propagation and back propagation.

6. A digital display method for water search and rescue exercises according to claim 4, characterized in that: In the step S123, the bilinear interpolation method is used to initialize the parameters of the output layer so that the center point weight of the convolution kernel of the output layer is the largest, and the edge weight decreases as the distance from the center increases; the padding and stride of the output layer are determined by the input size, convolution kernel size, padding and stride in the vertical and horizontal directions.

7. The digital display method for water search and rescue exercises according to claim 2, characterized in that: In the step S13, the specific steps are: S131: Determine the number of instances according to the types and numbers of participating units; S132: Create training labels for instance segmentation, establish a label matrix of the same size as the corresponding training image; assign a unique RGB value to the label of each participating unit, and establish a color mapping table; S133: The RGB color value of each pixel in the matrix, i.e., the pixel color of the label, is represented by the color mapping table and the background pixel of the non-participating unit instance; S134: Map the label matrix to the RGB image through a color mapping function.

8. The digital display method for water search and rescue exercises according to claim 1, characterized in that: In the step S2, the specific steps are: S21: Associating the mask point set and exercise information of the participating unit of each frame image of the video; S22: Matching the ID of the participating unit with the participating unit through the exercise information, and linking the exercise information with the corresponding participating unit; S23: Store the display information of each frame of the video into the database in the format of video frame time-participating unit ID-mask point set-exercise information.

9. A digital display system for water search and rescue exercises, characterized by: The video acquisition submodule is used to acquire exercise videos including participating units; The image recognition submodule is used to perform instance segmentation on the video content to identify the participating units in the video, obtain the mask point set of the participating units, and locate the position of the participating units in each frame of the video; The video processing and storage submodule is used to associate the mask point set of the identified participating units with the corresponding exercise information to form an operational link, embed the exercise information into the corresponding link, and store it in the database; A digital display submodule, used for digitally displaying the processed video and corresponding information; The operation response submodule is used to provide interactive operations, respond to operation inputs, and return results.

10. A computer memory, characterized in that: A computer program executable by a computer processor is stored therein, and the computer program executes a digital display method for water search and rescue exercises as described in any one of claims 1 to 8.

Images