Article expiration period management and rescue record intelligent interaction method of rescue carriage

Abstract

The invention relates to the technical field of image processing, and discloses an article expiration date management and rescue record intelligent interaction method for a rescue carriage, which comprises the following steps: a worker uses a terminal device to shoot an image of a name and a production date of an article, and automatically identifies the production date of the article by printing a date identification model; generating a rescue carriage article expiration date management table; during emergency treatment, the use condition of articles is recorded through voice, and oral medical advices are automatically recorded, so that workers can conveniently arrange the emergency treatment process after the emergency treatment is finished. The printing date recognition model constructed by the invention can quickly and accurately recognize the production date on the article, automatically detect the expiration date and actively remind, so that the energy of nurses is saved, medicines and articles are pre-processed, medical resource waste caused by expiration is avoided, and the life safety and medical quality of patients are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent interactive method for item validity management and rescue records of a rescue vehicle. Background Art

[0002] In emergency treatment, the expiration date management of items (including medicines and various items) in the rescue vehicle is extremely important, as it is directly related to the patient's life safety, medical quality, legal liability, and resource utilization efficiency. However, due to the wide variety of medicines and items in the rescue vehicle, the expiration date management is complicated, and the traditional manual inspection method is prone to omissions, resulting in the misuse of expired medicines and items, posing a threat to patient safety and causing waste of medical resources. In the prior art, although there are management systems for medicines and items in rescue vehicles, most of them are complex in function, inconvenient to operate, poor in flexibility and versatility, and expensive, making them inconvenient for medical institutions to choose to use.

[0003] At the same time, when rescuing patients in the rescue vehicle, nurses must not only follow the doctor's oral instructions to use the medication correctly, but also record the specific time of medication, drug name, medication method, dosage, etc., so as to write a rescue record after the rescue is over. This makes the tense rescue even more difficult to deal with. Summary of the invention

[0004] The purpose of the present invention is to improve the deficiencies in the prior art and to provide an intelligent interactive method for item validity management and rescue records of a rescue vehicle.

[0005] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0006] A method for intelligently interacting with the validity period management and rescue records of items in a rescue vehicle, comprising the following steps:

[0007] Step 1: The staff uses the terminal device to take an image of the name and production date of the item, automatically identifies the production date of the item through the printed date recognition model, and generates a rescue vehicle item validity management table;

[0008] The printing date recognition model includes a dual-stream network, a multi-stage self-attention network, and a multi-scale feature fusion network; the dual-stream network is used to extract text features and image features in the production date image, the multi-stage self-attention network is used to fuse the text features and image features through a multi-stage fusion method, and the multi-scale feature fusion network is used to perform multi-head attention calculation on the text features, image features, and fused features, so as to learn the feature representations between each other;

[0009] Step 2: During first aid, the use of items is recorded by voice, and oral medical instructions are automatically recorded, so that the staff can organize the first aid process after the first aid is completed.

[0010] Compared with the prior art, the present invention has the following beneficial effects: the printed date recognition model constructed by the present invention can quickly and accurately identify the production date on an item, and combined with a double entry mechanism, the production date input by the staff and the production date recognized by the model must be consistent before the number of days to the expiration date can be calculated, thus further ensuring the accuracy of the expiration date; the expiration date is automatically detected and proactively reminded, saving the nurses' energy, pre-processing medicines and items, avoiding waste of medical resources due to expiration, and ensuring the patient's life safety and medical quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 This is a schematic diagram of the structure of the printing date recognition model of the present invention;

[0013] Figure 2 It is a structural schematic diagram of a dual-stream network in the printing date recognition model of the present invention;

[0014] Figure 3 It is a schematic diagram of the structure of a multi-stage self-attention network in the printed date recognition model of the present invention;

[0015] Figure 4 A schematic diagram of identifying pixel color in the present invention;

[0016] Figure 5 Schematic diagram of the color change in the channel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0018] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0019] The present invention is achieved through the following technical solutions: Figure 1 As shown, a method for intelligent interaction of item validity management and rescue records of a rescue vehicle includes the following steps:

[0020] Step 1: The staff uses the terminal device to take an image of the name and production date of the item, automatically identifies the production date of the item through the printed date recognition model, and generates a rescue vehicle item expiration management table.

[0021] The placement of items in the rescue vehicle is generally completed by nurses. The items usually required in the rescue vehicle include medicines, syringes, gauze, bandages, cotton swabs, neck braces, gloves, masks, etc. Among them, the expiration date control of medicines is very critical. The problem of emergency patients using expired medicines must never occur. Therefore, this plan focuses on the medicines in the rescue vehicle as an example to introduce the technical solution.

[0022] The nurse first enters the information of the medicine into the terminal device, which can be any device such as a tablet or a mobile phone that can capture images and display screens, and the terminal device also has the functions of voice recording and voice playback. The nurse uses the terminal device to take a picture of the part of the medicine that displays the name, and then takes a picture of the part of the medicine that displays the production date, thereby generating a rescue vehicle item expiration management table. Usually, the shelf life of a drug is fixed, such as 3 years. Therefore, after the terminal device can recognize the name of the drug, it can automatically calculate the expiration date based on the recognized production date and organize it into a table to generate a rescue vehicle item expiration management table as shown in Table 1.

[0023] Table 1 Rescue vehicle item validity management table

[0024] The terminal device also classifies the medicine based on the efficacy of the medicine to place the medicines of the same category in the corresponding position, and displays in the rescue vehicle item expiration management table where a certain medicine should be placed in the medicine cabinet in the ambulance. As shown in Table 1, assuming that the medicine cabinet has three layers, each layer has three grids, the medicines can be classified according to the efficacy of the medicines according to cardiac resuscitation drugs, antiallergic drugs, analgesics, antiarrhythmic drugs, antihypertensive drugs, pressor drugs, sedatives and anticonvulsant drugs, antidotes, and other drugs, and each category of medicine is placed in a grid. For example, if the terminal device recognizes that a certain medicine is norepinephrine, it will classify the medicine as a cardiac resuscitation drug and prompt that it should be placed in the "first layer, first number" grid. It should be noted that the nurse can modify the classification method of the medicine according to the actual situation, such as classifying cardiac resuscitation drugs and pressor drugs into the same category. This embodiment is only used as an example and does not limit the specific classification method and placement position. For example, the placement method of cardiac resuscitation drugs can be based on the convenience of the nurse or doctor when rescuing.

[0025] If there are multiple items of the same drug, when the nurse enters the first item of a drug, the quantity can be modified directly in the table, so there is no need to take photos and enter the same drug multiple times. However, the nurse needs to strictly check whether the multiple drugs are from the same batch, that is, whether the production dates of the multiple drugs are on the same day. If they are different days, they need to be re-photographed and entered. Nurses can also take photos and enter the same drug multiple times. The system automatically accumulates data and automatically determines whether the production dates are the same. If the dates are different, the production date closest to the expiration date is used as a warning judgment.

[0026] This solution uses a printed date recognition model to automatically identify the name and production date of the drug. Since the names of the drugs are mostly in regular fonts, it is relatively easy to identify, but the display of the production date is usually not very obvious. The printing methods of drug production dates mainly include inkjet coding, laser coding, thermal transfer, embossing and printing technology, etc. In short, there are no other conspicuous words. Therefore, this application constructs a printed date recognition model for identifying the production date, improves its network structure and training process, and makes the printed date recognition model recognize the production date faster and more accurately.

[0027] After the terminal device captures the production date image of the drug, it inputs the printed date recognition model and first crops the image to keep only the image of the medicine box as much as possible and remove the background part relative to the medicine box. Figure 4 , each small grid in the cropped image represents a pixel ( Figure 4 This is just an example. In fact, the pixels are even smaller. The number "3" is the production date to be identified. Select any pixel (i, j) as the central node. i represents the index of the pixel row in the image, and j represents the index of the pixel column in the image. Figure 4The black frame is used to represent the color of the production date, and the white frame represents the color on the medicine box where the production date is located, which is called the background color.

[0028] Since the difference between the color of the production date and the background color is not obvious, the colors of all neighboring pixel points q of pixel point (i, j) are traversed. If the colors are the same as (i, j), pixel point q is added to the list corresponding to (i, j); if there are N pixels in the image, there will be N lists. Usually, the cropped image only has two colors: the production date and the background. Therefore, the lists are merged. If the color corresponding to any list is the same as the color corresponding to the other list, the two lists are merged and the same items are deleted. In this way, the number of lists can be reduced to at least two. For example, Figure 4 In the image shown, the pixels {(3,4), (3,5), (4,5), (5,4), (5,5), (6,5), (7,4), (7,5)} can be merged into one list, and the other pixels can be merged into another list. Then the colors in the same list are darkened in their channels, while the colors in the other list are lightened in their channels. This can make the color difference between the production date and the background larger, making it easier to identify. For example, Figure 5 As shown, a color becomes lighter as it moves to the left on the channel, and becomes darker as it moves to the right.

[0029] The printed date recognition model uses a convolutional neural network model as its basis to extract image features and text features and fuse them on the channel; then a multi-stage attention mechanism is added during the convolution and pooling process to encode image features, text features and fusion features to improve edge detection capabilities; finally, during the training process, an improved loss function is used to improve the model's color recognition capabilities, so that the model can accurately distinguish the color of the printed production date and the background color before recognition, which helps to improve recognition efficiency. For details, please refer to Figure 1 The printed date recognition model includes a two-stream network, a multi-stage self-attention network, and a multi-scale feature fusion network.

[0030] See also Figure 2, the dual-stream network includes a text stream and an image stream, wherein the text stream is used to extract text features from the original image, and the image stream is used to extract image features from the original image. Specifically, the text stream includes an LSTM unit and a first shallow feature extraction unit. After the original image is input into the LSTM unit, a text sequence is extracted, and then the text sequence is converted into a low-dimensional feature through the first shallow feature extraction unit, thereby obtaining a text feature V. The image stream includes a ResNet unit and a second shallow feature extraction unit. The original image is input into the ResNet unit to obtain a number of feature points, and then the number of feature points are converted into low-dimensional features through the second shallow feature extraction network, thereby obtaining an image feature E.

[0031] See also Figure 3 The multi-stage attention network includes a first-stage self-attention module, a second-stage self-attention module, ..., an L-th stage self-attention module connected in sequence, wherein the network structures from the second-stage self-attention module to the L-th stage self-attention module are the same.

[0032] The first-stage self-attention module includes a first convolution block, a first pooling layer, a first fusion layer, a second convolution block, a second pooling layer, a second fusion layer, and a first attention mechanism layer. The text feature V is input into the first convolution block to capture local features and map the input data to a higher-level feature space; it is then downsampled through the first pooling layer to reduce the amount of calculation and enhance the robustness of the model, thereby obtaining a feature vector V1; the text feature V and the feature vector V1 are then input into the first fusion layer together to obtain a feature vector F1. The image feature E is input into the second convolution block to capture local features (such as edges, textures, and shapes), and then downsampled through the second pooling layer to obtain a feature vector E1; the image feature E and the feature vector E1 are then input into the second fusion layer together to obtain a feature vector F2. The feature vector F1 and the feature vector F2 are input into the first attention mechanism layer together. The first attention mechanism layer has a multimodal Transformer function that can process text and images simultaneously to obtain a multimodal feature vector GF1.

[0033] The second-stage self-attention module includes a third convolution block, a third pooling layer, a fourth convolution block, a fourth pooling layer, and a second attention mechanism layer. The feature vector F1 is input into the third convolution block to capture local features, and then downsampled through the third pooling layer to obtain a feature vector F3. The feature vector F2 is input into the fourth convolution block to capture local features, and then downsampled through the fourth pooling layer to obtain a feature vector F4. The feature vector F3 and the feature vector F4 are input into the second attention mechanism layer together to obtain a multimodal feature vector GF2.

[0034] The number of stages of the self-attention module is preferably 4 to 6. Assuming that this embodiment uses a 4-stage self-attention module, the third-stage self-attention module includes a fifth convolution block, a fifth pooling layer, a sixth convolution block, a sixth pooling layer, and a third attention mechanism layer. The feature vector F3 is input into the fifth convolution block to capture local features, and then downsampled through the fifth pooling layer to obtain a feature vector F5. The feature vector F4 is input into the sixth convolution block to capture local features, and then downsampled through the sixth pooling layer to obtain a feature vector F6. The feature vector F5 and the feature vector F6 are input into the third attention mechanism layer together to obtain a multimodal feature vector GF3.

[0035] The fourth stage self-attention module includes the seventh convolution block, the seventh pooling layer, the eighth convolution block, the eighth pooling layer, and the fourth attention mechanism layer. The feature vector F5 is input into the seventh convolution block to capture the local features, and then downsampled through the seventh pooling layer to obtain the feature vector F7. The feature vector F6 is input into the eighth convolution block to capture the local features, and then downsampled through the eighth pooling layer to obtain the feature vector F8. The feature vector F7 and the feature vector F8 are input into the fourth attention mechanism layer together to obtain the multimodal feature vector GF4.

[0036] The multi-scale feature fusion network is used to encode the feature vectors obtained by the multi-stage self-attention network to strengthen and learn the relationship between text and image. Specifically, in the first stage self-attention module, the feature vector F1 is used as the key , taking the eigenvector F2 as the value , taking the multimodal feature vector GF1 as the query , respectively for the keys ,value and query Perform a flattening operation to convert a multidimensional vector into a one-dimensional vector , to facilitate multi-head attention calculation:

[0037] Among them, H1 represents the feature obtained after calculating the multi-head attention of the first-stage self-attention module; Represents the self-attention module in the first stage Query after flattening operation; Represents the self-attention module in the first stage The key after the flattening operation; Represents the self-attention module in the first stage The value after flattening; T represents matrix transpose; d1 represents the scaling factor of the first-stage self-attention module; softmax represents the activation function.

[0038] Similarly, the feature vector F2l-1 in the self-attention module of the first stage is used as the key , taking the eigenvector F2l as the value , taking the multimodal feature vector GFl as the query , respectively for the keys ,value and query Do the flattening operation and get , perform multi-head attention calculation:

[0039] Among them, H l Represents the features obtained after calculating the multi-head attention of the self-attention module in the lth stage; Represents the self-attention module in the first stage Query after flattening operation; Represents the self-attention module in the first stage The key after the flattening operation; Represents the self-attention module in the first stage The value after flattening; T represents matrix transpose; d l Represents the scaling factor of the self-attention module in the lth stage, which is a positive integer; l=1,2,...,L, where L is the number of stages of the self-attention module.

[0040] Finally, the features H1, H2, ...H obtained by the self-attention modules of L stages are L Perform multi-scale fusion:

[0041]

[0042] in, Represents the feature H of the self-attention module of L stages l The features obtained after multi-scale fusion, l=1,2,...,L; Reshape means restoring the dimension before flattening; Attn means multi-head attention fusion operation.

[0043] The multi-scale feature fusion network uses the prediction head to extract features In general, the printed date recognition model constructed in this scheme improves the loss function by adding color standard deviation and color variance to control the color variation degree of the target result and the background. The improved loss function is:

[0044] in, represents the variance loss function; N is the number of pixels in the original image, N1 is the number of pixel rows in the original image, N2 is the number of pixel columns in the original image, and N1×N2=N, i is the index of the row, j is the index of the column, k is the index of the row except i, and h is the index of the column except j; p i,j Represents the category probability of pixel (i, j); represents the variance of the true result x, Represents the variance of the predicted result y; is a constant greater than 0 and less than or equal to 1, used to avoid the denominator being zero; represents the standard deviation loss function; Represents the covariance between the true result x and the predicted result y; represents the standard deviation of the true result x, Represents the standard deviation of the predicted result y; is the total loss function; for The weight of for The weight of .

[0045] In the improved loss function of this solution, the variance and standard deviation of the color are constrained. The larger the variance, the larger the pixel edge smoothing range and the less detail is retained; the larger the standard deviation, the higher the tolerance for color differences and the higher the edge blur. Therefore, by finding the optimal color change degree, the model can change the color difference between the target and the background without blurring the edge of the target and can clearly identify the target.

[0046] In order to ensure that the expiration date of the drug is accurate, this solution adopts a double entry mechanism. After taking a photo of the name and production date of the drug, it is automatically organized into the rescue vehicle item expiration date management table as shown in Table 1. However, the type, location, name, and quantity of the drug are displayed first. The nurse needs to manually enter the production date in the "Production Date" column, and then the system determines whether the production date entered by the nurse is consistent with the production date identified by the photo. If they are consistent, the "number of days to expiration date" is directly calculated; if they are inconsistent, a prompt box will pop up saying "Please check again whether the production date is xxxx-xx-xx", with "Correct" and "Modify" selection boxes attached. The nurse will check the production date on the drug again. If the nurse fills in the wrong information, select "Modify" and re-enter it in the "Production Date" column. If the nurse does not fill in the right information, it means that the photo identification is wrong, then select "Correct", and then directly calculate the "number of days to expiration date".

[0047] It can be seen that this system adopts a double entry mechanism to ensure the accuracy of the drug expiration date. If the nurse fills in the production date correctly, but the production date recognized by the photo is wrong, the system will use the photo taken and the correct production date as training data to retrain the model to improve recognition accuracy.

[0048] After obtaining the "days to expiration", if there are more than two production dates for the same drug, the drug with shorter days to expiration will be placed outside the grid, and the drug with longer days to expiration will be placed inside the grid, so that the drugs that are relatively close to expiring can be used first.

[0049] In the rescue vehicle item expiration management table, mark the drugs with "days to expiration" less than 180 days, and nurses need to put conspicuous labels on the outside of these drugs to remind nurses or doctors to pay special attention to the drugs. If the "days to expiration" is less than 7 days, a prompt box "Please discard XX drug produced by xxxx-xx-xx" will pop up, with a "discarded" selection box attached. Then the nurse needs to discard these drugs and click "discarded", and then the monitoring of these drugs will be automatically deleted from the rescue vehicle item expiration management table.

[0050] Step 2: During first aid, the use of items is recorded by voice, and oral medical instructions are automatically recorded, so that the staff can organize the first aid process after the first aid is completed.

[0051] In the terminal device, the rescue and transportation process is recorded in real time through intelligent voice recording and sorting. For example, if the doctor wants to inject 1mL of norepinephrine to the patient, the nurse will wake up the voice recording through specific words, and record the words spoken by the nurse during the first aid process through natural language processing (NLP) technology. The purpose of sorting is to automatically remove repeated words. For example, in a tense situation, there may be stuttering words, so the repeated words caused by stuttering are deleted. However, repeated sentences are not deleted, because it is very likely that the operation will be repeated many times during the first aid process, such as "200 joules once", and "200 joules once" again after 1 minute. Such repeated operations must be recorded, and the time must be matched when recording, accurate to seconds. In this way, after the first aid process of the ambulance is over, the nurse can sort it again according to the record to form the first aid record stipulated by the hospital. After the first aid record is formed, the quantity of medicines and items in the expiration management table of the rescue vehicle items will also be updated.

[0052] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for intelligent interaction of item validity management and rescue records of a rescue vehicle, characterized by: The following steps are involved: Step 1: The staff uses the terminal device to take an image of the name and production date of the item, automatically identifies the production date of the item through the printed date recognition model, and generates a rescue vehicle item validity management table; The printing date recognition model includes a dual-stream network, a multi-stage self-attention network, and a multi-scale feature fusion network; the dual-stream network is used to extract text features and image features in the production date image, the multi-stage self-attention network is used to fuse the text features and image features through a multi-stage fusion method, and the multi-scale feature fusion network is used to perform multi-head attention calculation on the text features, image features, and fused features, so as to learn the feature representations between each other; Step 2: During first aid, the use of items is recorded by voice, and oral medical instructions are automatically recorded, so that the staff can organize the first aid process after the first aid is completed.

2. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 1, characterized in that: In step 1, when the printing date recognition model recognizes the production date of an item, it first crops the image and selects any pixel point (i, j) as the central node, where i represents the index of the pixel row in the image and j represents the index of the pixel column in the image; Traverse the colors of all neighboring pixel points q of pixel point (i, j). If the colors are the same as those of (i, j), add pixel point q to the list corresponding to (i, j). Assuming there are N pixels in the image, there will be N lists. The lists are merged. If the color corresponding to any list is the same as the color corresponding to the other list, the two lists are merged and the same items are deleted. Then the colors in the same list are darkened in its channel, while the colors in the other list are lightened in its channel, so that the color difference between the production date and the background becomes larger.

3. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 1, characterized in that: The rescue vehicle item validity management table includes the placement location, name, production date, and number of days until expiration; after identifying the item name, the items are classified according to their efficacy and the placement location of the items is displayed; The staff manually inputs the production date of the item. If the manually input production date is consistent with the production date recognized by the printed date recognition model, it will be displayed in the production date column of the rescue vehicle item validity management table; if they are inconsistent, the staff will be prompted to re-determine the production date; after determining the production date, the number of days to the expiration date will be automatically calculated.

4. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 1, characterized in that: The dual-stream network includes a text stream and an image stream, wherein the text stream includes an LSTM unit and a first shallow feature extraction unit. After the original image is input into the LSTM unit, a text sequence is extracted, and then the text sequence is converted into a low-dimensional feature through the first shallow feature extraction unit, thereby obtaining a text feature V; the image stream includes a ResNet unit and a second shallow feature extraction unit. The original image is input into the ResNet unit to obtain a number of feature points, and then the number of feature points are converted into low-dimensional features through the second shallow feature extraction network, thereby obtaining an image feature E.

5. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 4, characterized in that: The multi-stage attention network includes a first-stage self-attention module, a second-stage self-attention module, ..., an L-th-stage self-attention module connected in sequence, wherein the network structures of the second-stage self-attention module to the L-th-stage self-attention module are the same; The first-stage self-attention module includes the first convolution block, the first pooling layer, the first fusion layer, the second convolution block, the second pooling layer, the second fusion layer, and the first attention mechanism layer; the text feature V is input into the first convolution block to capture the local features, and then downsampled through the first pooling layer to obtain the feature vector V1; the text feature V and the feature vector V1 are input into the first fusion layer together to obtain the feature vector F1; the image feature E is input into the second convolution block to capture the local features, and then downsampled through the second pooling layer to obtain the feature vector E1; the image feature E and the feature vector E1 are input into the second fusion layer together to obtain the feature vector F2; Input the feature vector F1 and the feature vector F2 into the first attention mechanism layer to obtain the multimodal feature vector GF1; The self-attention module in the first stage includes the 2l-1th convolution block, the 2l-1th pooling layer, the 2l-1th convolution block, the 2l-1th pooling layer, and the 1st attention mechanism layer; the feature vector F2l-3 is input into the 2l-1th convolution block to capture the local features, and then downsampled through the 2l-1th pooling layer to obtain the feature vector F2l-1; The feature vector F2l-2 is input into the 2lth convolutional block to capture local features, and then downsampled through the 2lth pooling layer to obtain the feature vector F2l; the feature vector F2l-1 and the feature vector F2l are input into the lth attention mechanism layer together to obtain the multimodal feature vector GFl; l=2,3,...,L, L is the number of stages of the self-attention module.

6. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 5, characterized in that: The multi-scale feature fusion network uses the feature vector F2l-1 in the first stage self-attention module as the key , taking the eigenvector F2l as the value , taking the multimodal feature vector GFl as the query , respectively for the keys ,value and query Do the flattening operation and get , perform multi-head attention calculation: Among them, H l Represents the features obtained after calculating the multi-head attention of the self-attention module in the lth stage; Represents the self-attention module in the first stage Query after flattening operation; Represents the self-attention module in the first stage The key after the flattening operation; Represents the self-attention module in the first stage The value after flattening; T represents matrix transpose; d l represents the scaling factor of the self-attention module at the lth stage; l=1,2,...,L, where L is the number of stages of the self-attention module; Finally, the features H1, H2, ...H obtained by the self-attention modules of L stages are L Perform multi-scale fusion: in, Represents the feature H of the self-attention module of L stages l The features obtained after multi-scale fusion, l=1,2,...,L; Reshape means restoring the dimension before flattening; Attn means multi-head attention fusion operation.

7. The method for intelligently interacting with the item validity period management and rescue record of a rescue vehicle according to claim 6, characterized in that: The loss function of the printing date recognition model is: in, represents the variance loss function; N is the number of pixels in the original image, N1 is the number of pixel rows in the original image, N2 is the number of pixel columns in the original image, and N1×N2=N, i is the index of the row, j is the index of the column, k is the index of the row except i, and h is the index of the column except j; p i,j Represents the category probability of pixel (i, j); represents the variance of the true result x, Represents the variance of the predicted result y; is a constant; represents the standard deviation loss function; Represents the covariance between the true result x and the predicted result y; represents the standard deviation of the true result x, Indicates the standard deviation of the predicted result y; is the total loss function; for The weight of for The weight of .

Images