Space station material management system and method based on multi-dimensional feedback information

Abstract

The invention discloses a space station material management system and method based on multi-dimensional feedback information, and belongs to the technical field of space station material management. An on-orbit end of the system comprises a multi-dimensional material information acquisition module, a material operation instruction acquisition module and an on-orbit material management computing node; the multi-dimensional material information acquisition module comprises any combination of a first video and image acquisition device, a first audio acquisition device, a first text acquisition device, a label acquisition device and a two-dimensional code acquisition device. And the ground material management computing node at the ground end comprises a material business processing module and a feedback business processing module. In view of a space station material management scene, a multi-dimensional material information acquisition mode is provided, a worker can select a material information acquisition mode better meeting own requirements in the current scene, and the requirements of space material acquisition and operation instruction acquisition can be met; and the material information and the material operation instruction are calculated and sent to the ground for processing, so that effective space station material management is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of space station material management, and in particular to a space station material management system and method based on multi-dimensional feedback information. Background Art

[0002] The material information management method is widely used in computer systems. For example, the invention patent application with publication number CN118644180A and patent name "A material management method and system based on QR code" uses QR code to identify materials, combines communication technology and database management, and realizes intelligent management of the entire process of material entry, exit and inventory; for example, the invention patent application with publication number CN118711775A and "An intelligent insulin pump management device and application based on RFID technology" uses intelligent storage technology to intelligently store and store insulin pumps and their consumables. Based on RFID technology, each time a user stores and uses insulin pumps and their consumables, the intelligent storage cabinet records each time a user stores and uses insulin pumps and consumables by swiping the card and reports it to the system. The background server can supervise the storage, replenishment, supply, etc. of the stored insulin pumps and their consumables based on the reported intelligent storage management information. In addition, with regard to the material management of the space station, an invention patent with publication number CN107871147A and patent name "Space Station Material Management System and Management Method Thereof" discloses a material management method based on barcode scanning. By querying the information stored in the main control computer that indicates the entry and exit of items in the space station, the transfer and consumption in the space station, the entry, exit, transfer and inventory information of the items are obtained, thereby realizing the space station material management.

[0003] At present, the material information management method is mainly used in ground scenarios. In the material information management system used in space vehicles such as space stations, the main collection method based on identification methods such as QR codes or bar codes is currently used for material management. However, the types of materials in the space station are diverse, and not all materials can be attached with QR codes. It is difficult to meet all needs using a single method. In addition, due to the restrictions on astronauts' activities in orbit in some cases, it is difficult to achieve effective management using conventional ground material management methods such as scanning codes in some scenarios. Summary of the invention

[0004] The purpose of the present invention is to overcome the problems of the prior art and to provide a space station material management system and method based on multi-dimensional feedback information.

[0005] The object of the present invention is to achieve the following technical solutions: a space station material management system based on multi-dimensional feedback information, the system includes an orbital end and a ground end; the orbital end includes a multi-dimensional material information acquisition module, a material operation instruction acquisition module and an on-orbit material management computing node, the multi-dimensional material information acquisition module includes any combination of a first video and image acquisition device, a first audio acquisition device, a first text acquisition device, a label acquisition device, and a two-dimensional code acquisition device, which is used to collect material information, and the material is provided with a label and / or a two-dimensional code; the material operation instruction acquisition module is any one or more combinations of a second video and image acquisition device, a second audio acquisition device, and a second text acquisition device, which is used to collect material operation instructions; the on-orbit material management computing node is used to solve the material information and the material operation instruction to obtain material result information and material operation information, and the material result information includes material name, material location information, material shape and type, material number, and material state information;

[0006] The ground end includes a distributed operation center, which includes a ground material management computing node. The ground material management computing node includes a material business processing module and a feedback business processing module. The material business processing module is used to classify and merge materials and material operations according to material result information and material operation information, and generate material management business information, including the operation methods of each material, the users who perform material operations, and the processing deadline; the feedback business processing module is used to perform business processing according to the material management business information, and then feed back the business processing results to the on-orbit end.

[0007] In one example, the on-orbit end also includes a mobile material management device, a mobile material management system is integrated on the mobile material management device, and an on-orbit material information management system is integrated on the on-orbit material management computing node.

[0008] In one example, the on-track end also includes an access control system for managing the entry and exit of materials and movement records.

[0009] In one example, a ground material information management system is integrated on the ground material management computing node.

[0010] In one example, the ground material management computing node also includes a data solving module, which is used to receive and solve material information and material operation instructions when the on-orbit material management computing node is unable to solve the data results.

[0011] It should be further explained that the technical features corresponding to the above system examples can be combined or replaced with each other to form a new technical solution.

[0012] The present invention also includes a space station material management method based on multi-dimensional feedback information, the method comprising the following steps:

[0013] Multi-dimensional material information collection and material operation instructions; the multi-dimensional material information collection module includes any combination of a first video and image collection device, a first audio collection device, a first text collection device, a label collection device, and a QR code collection device;

[0014] Solve the material information and material operation instructions to obtain material result information and material operation information. The material result information includes material name, material location information, material shape and type, material number, and material status information;

[0015] Classify and merge materials and material operations according to material result information and material operation information to generate material management business information, including the operation method of each material, the user who performs the material operation, and the processing deadline;

[0016] Business processing is carried out according to the material management business information, and then the business processing results are fed back to the on-track end.

[0017] In one example, after obtaining the material result information and the material operation information, the following is further included:

[0018] Correct the material result information and material operation information, and mark and remind them according to the urgency of the processing.

[0019] In one example, before performing business processing and updating according to the material management business information, the following steps are also included:

[0020] Distribute material management business information to users who are responsible for performing material operations for processing.

[0021] In one example, after business processing is performed according to the material management business information, the following is further included:

[0022] Update corresponding materials based on business processing results.

[0023] In one example, after the corresponding materials are updated, the following is also included:

[0024] Confirm the results of business processing and material update.

[0025] It should be further explained that the technical features corresponding to the various examples of the above method can be combined or replaced with each other to form a new technical solution.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. In one example, for the space station material management scenario, the present invention provides a multi-dimensional material information collection method. The workstation staff can choose the information collection form that better meets their own requirements in the current scenario to collect material information and material operation instructions, providing the space station staff with an optional and more convenient material information collection method that can meet the requirements of space material collection and operation instruction collection, solve the material information and material operation instructions and send them to the ground for processing, thereby realizing effective space station material management.

[0028] 2. In one example, by setting up on-orbit material management computing nodes and / or mobile material management equipment, space materials can be further managed, making it easier for space station staff to quickly obtain relevant material information, including material name, material operation history, material number, material status information, material shape and type, material location information, etc.

[0029] 3. In one example, by introducing an access control system to further manage the entry, exit and movement information of materials, the information of materials entering and leaving the warehouse can be effectively obtained, providing effective data support for material management.

[0030] 4. In one example, a ground material information management system is used to synchronize and manage materials, making it easier for ground staff to obtain relevant material information.

[0031] 5. In one example, due to the high network latency of the space station and the inability to use high-power large-scale intelligent solving servers, the data solving capabilities of the on-orbit material management computing nodes configured on the space station are limited. By introducing high-power large-scale intelligent solving servers, namely data solving modules, on the ground, material information and material operation instructions can be identified more accurately and quickly, thereby improving the effectiveness and reliability of material management.

[0032] 6. In one example, correcting the material result information and material operation information can further ensure the accuracy of the material result information and material operation information and improve the accuracy of material management.

[0033] 7. In one example, after performing material operations and corresponding material updates, reconfirming the business processing results and material update results can improve the reliability of material processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings. The accompanying drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0035] Figure 1 A block diagram of a material management system provided as an example of the present invention;

[0036] Figure 2 A schematic diagram of a target detection mode of an existing frame difference method;

[0037] Figure 3 A flow chart of a binocular double difference algorithm provided as an example of the present invention;

[0038] Figure 4 A schematic diagram of a target detection mode of the binocular double difference method of the present invention;

[0039] Figure 5 A comparison diagram of target detection modes between the binocular double difference method of the present invention and the existing frame difference method;

[0040] Figure 6 A block diagram of a material management system provided for a preferred example of the present invention;

[0041] Figure 7 A flow chart of a material management method provided as an example of the present invention;

[0042] Figure 8 A flow chart of a material management method provided for a preferred example of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In the description of the present invention, it should be noted that the directions or positional relationships indicated by "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. are directions or positional relationships based on the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the use of ordinal numbers (e.g., "first and second", "first to fourth", etc.) is to distinguish objects, and is not limited to this order, and cannot be understood as indicating or implying relative importance.

[0045] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0047] In one example, a space station material management system based on multi-dimensional feedback information, such as Figure 1 As shown, the system includes an on-orbit end and a ground end, and the on-orbit end and the ground end communicate with each other through a space-ground network. The on-orbit end includes a multi-dimensional material information acquisition module, a material operation instruction acquisition module, and an on-orbit material management computing node. The ground end includes a distributed operation center, which includes a ground material management computing node, and the ground material management computing node includes a material business processing module and a feedback business processing module.

[0048] Further, the multi-dimensional material information acquisition module is used to collect material information, including any combination of the first video and image acquisition device, the first audio acquisition device, the first text acquisition device, the label acquisition device, and the two-dimensional code acquisition device. Preferably, the multi-dimensional material information acquisition module includes the first video and image acquisition device, the first audio acquisition device, the first text acquisition device, the label acquisition device, and the two-dimensional code acquisition device. At this time, the material is provided with a label and a two-dimensional code, which is convenient for the label acquisition device and the two-dimensional code acquisition device to read the label information and the two-dimensional code information on the material respectively, and then collect the material information, and then transmit it to the on-orbit material management and calculation node for material solution processing. More preferably, the label acquisition device and the two-dimensional code acquisition device can directly parse the collected material information. At this time, only the material information collected by the first video and image acquisition device, the first audio acquisition device, and the first text acquisition device needs to be transmitted to the on-orbit material management and calculation node for solution.

[0049] Further, the material operation instruction collection module is used to collect material operation instructions, including taking, transporting, counting, returning to the warehouse, abandoning, installing, registering, etc. The material operation instruction collection module is any one or more combinations of the second video and image acquisition device, the second audio acquisition device, and the second text acquisition device, preferably the second audio acquisition device or the second text acquisition device. At this time, the space station staff can issue material operation instructions by inputting material operation instructions (voice instructions) to the second audio acquisition device or inputting material operation instructions (text instructions) to the second text acquisition device. Optionally, in order to save system cost, the first video and image acquisition device and the second video and image acquisition device are the same device, the first audio acquisition device and the second audio acquisition device are the same device, and the first text acquisition device and the second text acquisition device are the same device. Preferably, a device that integrates video and image input, text input, and voice input can be selected to replace the above-mentioned first video and image acquisition device, the second video and image acquisition device, the first audio acquisition device, the second audio acquisition device, the first text acquisition device, and the second text acquisition device, such as a mobile terminal, such as a mobile phone, a tablet, etc.

[0050] Furthermore, the on-orbit material management computing node can be an on-orbit material management computer, which is used to solve the material information and material operation instructions to obtain material result information and material operation information, and the material result information includes material name, material location information, material shape and type, material number, and material state information. Further, the on-orbit material management computing node collects and standardizes data according to the solved material information type, feedback person, feedback time, feedback content, material information, operation information, material location information, and attachments (audio, graphics, video). Preferably, before the on-orbit material management computing node solves the material information and material operation instructions, it also includes pre-processing the video stream, image data, audio data, and text data to remove interference information. The on-orbit material management computing node includes an image solving module, a voice solving module, a text solving module, a label solving module and a two-dimensional solving module. Preferably, the on-orbit material management computing node includes an image solving module, a voice solving module and a text solving module. The image solving module can be a video server for solving the collected video stream and image data (including video streams and image data of material information and material operation instructions). In the present invention, the image solving module designs a dynamic target detection of binocular double difference method. Different from the traditional ground materials being affected by gravity and showing a fixed state, the material management method of the present invention is applied to environments such as space stations. The materials are in weightless conditions and may be in a floating and moving non-fixed state and artificially moving situations, so it is necessary to perform dynamic detection on the target. At present, the target detection of other scenes on the ground generally uses a monocular camera frame difference method to solve the image information, by performing a differential operation on two or more consecutive frames of images in the sequence, and then thresholding the differential image. If it is greater than the set threshold, it is marked as a foreground pixel, and finally the target is discriminated to obtain a moving target. The monocular camera frame difference method has a simple implementation principle, is easy to implement, and has low computational complexity. However, when the frame difference method is used to detect targets moving along the camera axis in a video, Figure 2 As shown, because the pixel values ​​of the corresponding positions in adjacent frames of the central area are similar, the area is identified as the background, and a "hole" phenomenon appears inside the target in the generated binary image. When the movement speed of the foreground target is too fast relative to the camera's shooting frame rate, a long "smear" will be generated at the tail of the target. In order to solve the problem of edge black holes formed when the camera moves in the X-axis line of sight, the image solution module of the present invention adopts binocular double difference method for dynamic target detection. For the camera resolution of 1920*1080, compression ratio of 0.75, video frame rate of 30fps, after image preprocessing and normalization, the distance h between the first camera (camera A) and the second camera (camera B) is in the range of 30cm to 50cm, as shown in FIG. Figure 3 The detection methods include:

[0051] Performing differential processing on the current frame image captured by the first camera and the previous frame image to obtain a first differential image (differential image Ua); performing differential processing on the current frame image captured by the second camera and the previous frame image to obtain a second differential image (differential image Ub);

[0052] Performing threshold processing and binarization processing on the first difference image and the second difference image respectively, to obtain a first binarized image and a second binarized image respectively;

[0053] The first binary image and the second binary image are subjected to double difference processing to obtain a final difference image (difference image U), and then the final difference image is subjected to threshold processing to obtain a binary image of the foreground target.

[0054] Correspondingly, the calculation expression of the binocular double difference method is:

[0055] R n (x,y)=|f A(n) (x,y)-f A(n-1) (x,y)|-|f B(n) (x,y)-f B(n-1) (x,y)|

[0056] Among them, f A(n) (x,y),f A(n-1) |(x,y) are the image pixel values ​​of the current frame and the previous frame in the video sequence of camera A respectively; f B(n) (x,y),f B(n-1) (x, y) are the image pixel values ​​of the current frame and the previous frame in the video sequence of camera B respectively; R n (x, y) represents the foreground image of the moving target. The double difference method of the present invention obtains two difference images by performing adjacent difference operations on the four images respectively, and then performs "double difference" on the two images. Only pixels that are all identified as foregrounds will be extracted, and then binarized according to the set threshold to obtain the foreground image of the moving target.

[0057] like Figure 4 As shown in FIG. 1 , for a moving target on the axis, the target features calculated by the binocular double difference method of the present invention are more complete, the area is larger, and the contour details are more completely preserved, which can solve the "hole problem" of the traditional monocular frame difference method to a certain extent. More specifically, the holes calculated by the existing monocular frame difference method are as follows: Figure 5 As shown in (a), the hole calculated by the binocular double difference method is as follows Figure 5 Furthermore, the existing monocular camera can only obtain the features of one plane in the vertical viewing angle, while the binocular double difference method of the present invention can obtain the features of two or more planes of the target, thereby obtaining richer and more complete target feature information, thereby improving the accuracy of target detection.

[0058] Furthermore, the speech solution module includes a speech-to-text unit and a text recognition unit. The speech-to-text unit is used to convert the collected user's voice instructions (including material information and material operation instructions) into text information. The text recognition unit is used to perform word segmentation and semantic analysis on the text information to obtain semantic parsing data, such as the recognition and deep understanding of text information based on the transformer model. The text solution module is used to perform word segmentation and semantic analysis on the collected text data (including material information and material operation instructions) to obtain semantic parsing data, such as the recognition and deep understanding of text information based on the transformer model.

[0059] Further, the material business processing module is used to classify and merge materials and material operations according to the material result information and material operation information, and generate material management business information, including the operation method of each material, the user who performs the material operation, and the processing deadline. Specifically, the material business processing module classifies the related material information, merges the material information with the same operation, and classifies the material information and operations of the same handler; automatically generates material management business (including material information, processing methods, handlers, processing deadlines, etc.) and associates it with the corresponding feedback. Preferably, before the material business processing module classifies and merges materials and material operations, it also includes manual auxiliary annotation processing, that is, viewing text, pictures and voice, editing and error correction of the content of automatic recognition and analysis, and annotating key information (such as operation information, etc.) through tags, and configuring colors according to the degree of urgency. Further, the feedback business processing module is used to perform business processing according to the material management business information, and then feedback the business processing results to the on-track end. Preferably, the feedback business processing module manually processes the business that needs to be manually processed and submits it after the processing is completed.

[0060] This example is aimed at the space station material management scenario. The present invention provides a multi-dimensional material information collection method. The workstation staff can choose the information collection form that better meets their own requirements in the current scenario to collect material information and material operation instructions, providing the space station staff with an optional and more convenient information collection method that can meet the requirements of space material collection and operation instruction collection, solve the material information and material operation instructions and send them to the ground for processing, thereby realizing effective space station material management.

[0061] In one example, if Figure 6As shown, the on-orbit end also includes mobile material management equipment, on which a mobile material management system is integrated, and on-orbit material management computing nodes are integrated with an on-orbit material information management system. Both the mobile material management system and the on-orbit material information management system are used to manage space station materials and store material information, such as material name, material operation history, material number, material status information, material shape and type, material location information, etc., to facilitate staff to trace or obtain material information in a timely manner.

[0062] In one example, the rail end also includes an access control system for managing the entry and exit and movement records of materials. Optionally, the access control system can be an access control system based on tags (FID tags) for recording each time the materials enter and leave the warehouse. In this case, the multi-dimensional material information collection module on the rail end does not need to be additionally configured with a tag collection device, that is, the tag collection device set on the access control system can simultaneously obtain material information and RFID records (RFID entry and exit, movement records) when entering and leaving the warehouse, thereby further saving system cost overhead.

[0063] In one example, a ground material management computing node is integrated with a ground material information management system, and the ground material management computing node and the on-orbit material management computing node communicate with each other via the space-ground network relay, thereby achieving data synchronization between the ground material information management system and the on-orbit material information management system, making it easier for ground staff to obtain relevant material information, and then cooperate with the space station staff to perform corresponding business operations on the corresponding materials.

[0064] In one example, the ground material management computing node also includes a data solution module, which is used to receive and solve material information and material operation instructions when the on-orbit material management computing node cannot solve the data results. Due to the high network delay of the space station and the inability to use a high-power large-scale intelligent solution server, the data solution capability of the on-orbit material management computing node configured on the space station is limited. By introducing a high-power large-scale intelligent solution server, that is, a data solution module, the data solution module includes an image solution submodule, a voice solution submodule, a text solution submodule, a label solution submodule and a two-dimensional solution submodule. Preferably, the on-orbit material management computing node includes an image solution submodule, a voice solution submodule, and a text solution submodule, and the hardware configuration performance and software (model) settlement capability corresponding to the data solution adopted by each submodule are due to the on-orbit material management computing node, so that the material information and material operation instructions can be more accurately and quickly identified, thereby improving the effectiveness and reliability of material management and providing corresponding guarantees for the data solution of the space station.

[0065] Preferably, the above examples are combined to obtain a preferred example of the present invention as follows Figure 6As shown, the system includes the on-orbit end and the ground end. The on-orbit end includes a multi-dimensional material information acquisition module, a material operation instruction acquisition module, an on-orbit material management computing node, a mobile material management device and an access control system. The on-orbit material management computing node is integrated with an on-orbit material information management system, and the mobile material management device is integrated with a mobile material management system. The ground end includes a distributed operation center, which includes a ground material management computing node, and the ground material management computing node includes a material business processing module, a feedback business processing module and a data solution module.

[0066] The present invention also includes a space station material management method based on multi-dimensional feedback information, which has the same inventive concept as the above-mentioned space station material management system, such as Figure 7 As shown, the method comprises the following steps:

[0067] S1: Multi-dimensionally collect material information and material operation instructions; the multi-dimensional material information collection module includes any combination of a first video and image collection device, a first audio collection device, a first text collection device, a label collection device, and a QR code collection device.

[0068] Preferably, the multi-dimensional collection of material information is implemented based on a multi-dimensional material information collection module, which includes a first video and image collection device, a first audio collection device, a first text collection device, a label collection device, and a QR code collection device. At this time, labels and QR codes are correspondingly provided on the materials, so that the label collection device and the QR code collection device can read the label information and the QR code information on the materials respectively, and then collect the material information, and then perform material solution processing. More preferably, the label collection device and the QR code collection device can directly parse the collected material information. At this time, only the material information collected by the first video and image collection device, the first audio collection device, and the first text collection device needs to be transmitted to the on-orbit material management and computing node for solution. Preferably, a device that integrates video and image input, text input, and voice input can be selected to collect material information and material operation instructions at the same time.

[0069] S2: Calculate the material information and material operation instructions to obtain material result information and material operation information. The material result information includes material name, material location information, material shape and type, material number, material status information, and material operation instructions, including taking, transferring, counting, returning to the warehouse, discarding, installing, registering, etc.

[0070] Preferably, before the on-orbit material management computing node solves the material information and material operation instructions, it also includes pre-processing the video stream, image data, audio data, and text data to remove interference information. The on-orbit material management computing node is used to perform image solving, voice solving, text solving, label solving and QR code solving. Preferably, the on-orbit material management computing node performs image solving, voice solving and text solving. The present invention designs dynamic target detection based on binocular double difference method, that is, the collected video stream and image data (including video stream and image data of material information and material operation instructions) are solved. Different from traditional ground materials that are fixed under the influence of gravity, the material management method of the present invention is applied to environments such as space stations. The materials are in weightless conditions and may be in a non-fixed state of floating movement and artificial movement. Therefore, it is necessary to perform dynamic detection of the target. At present, target detection in other scenes on the ground generally uses the monocular camera frame difference method to solve the image information. It performs a difference operation on two or more consecutive frames in the sequence, and then thresholds the difference image. If it is greater than the set threshold, it is marked as a foreground pixel, and finally the target is identified to obtain the moving target. The monocular camera frame difference method has a simple implementation principle, is easy to implement, and has low computational complexity. However, when the frame difference method is used to detect targets moving along the camera axis in the video, such as Figure 2 As shown, because the pixel values ​​of the corresponding positions in adjacent frames of the central area are similar, the area is identified as the background, and a "hole" phenomenon appears inside the target in the generated binary image. When the movement speed of the foreground target is too fast relative to the camera's shooting frame rate, a long "smear" will be generated at the tail of the target. In order to solve the problem of edge black holes formed when the camera's X-axis line of sight moves in the direction of the dynamic target, the image solution module of the present invention uses a binocular double difference method to detect dynamic targets. For a camera resolution of 1920*1080, a compression ratio of 0.75, and a video frame rate of 30fps, after image preprocessing and normalization, the distance h between the first camera (camera A) and the second camera (camera B) is in the range of 30cm to 50cm. As shown Figure 3 As shown, the detection method includes:

[0071] Performing differential processing on the current frame image captured by the first camera and the previous frame image to obtain a first differential image (differential image Ua); performing differential processing on the current frame image captured by the second camera and the previous frame image to obtain a second differential image (differential image Ub);

[0072] Performing threshold processing and binarization processing on the first difference image and the second difference image respectively, to obtain a first binarized image and a second binarized image respectively;

[0073] The first binary image and the second binary image are subjected to double difference processing to obtain a final difference image (difference image U), and then the final difference image is subjected to threshold processing to obtain a binary image of the foreground target.

[0074] Correspondingly, the calculation expression of the binocular double difference method is:

[0075] R n (x,y)=|f A(n) (x,y)-f A(n-1) (x,y)|-|f B(n) (x,y)-f B(n-1) (x,y)|

[0076] Among them, f A(n) (x,y),f A(n-1) (x, y) are the image pixel values ​​of the current frame and the previous frame in the video sequence of camera A respectively; f B(n) (x,y),f B(n-1) (x, y) are the image pixel values ​​of the current frame and the previous frame in the video sequence of camera B respectively; R n (x, y) represents the foreground image of the moving target. The double difference method of the present invention obtains two difference images by performing adjacent difference operations on the four images respectively, and then performs "double difference" on the two images. Only pixels that are all identified as foregrounds will be extracted, and then binarized according to the set threshold to obtain the foreground image of the moving target.

[0077] like Figure 4 As shown in FIG. 1 , for a moving target on the axis, the target features calculated by the binocular double difference method of the present invention are more complete, the area is larger, and the contour details are more completely preserved, which can solve the "hole problem" of the traditional monocular frame difference method to a certain extent. More specifically, the holes calculated by the existing monocular frame difference method are as follows: Figure 5 As shown in (a), the hole calculated by the binocular double difference method is as follows Figure 5 Furthermore, the existing monocular camera can only obtain the features of one plane in the vertical viewing angle, while the binocular double difference method of the present invention can obtain the features of two or more planes of the target, thereby obtaining richer and more complete target feature information, thereby improving the accuracy of target detection.

[0078] Furthermore, the speech solution step includes two steps: speech-to-text and text recognition. Speech-to-text converts the collected user's voice instructions (including material information and material operation instructions) into text information. Text recognition performs word segmentation and semantic analysis on the text information to obtain semantic analysis data. For example, the recognition and deep understanding of text information can be realized by using a transformer-based model. In the text solution step, the collected text data (including material information and material operation instructions) is subjected to word segmentation and semantic analysis to obtain semantic analysis data. For example, the recognition and deep understanding of text information can be realized by using a transformer-based model.

[0079] S3: Classify and merge materials and material operations according to material result information and material operation information to generate material management business information, including the operation method of each material, the user who performs the material operation, and the processing deadline.

[0080] Classify and consolidate materials and material operations, including:

[0081] Classify related material information, merge material information with similar operations, and classify material information and operations with the same handler; automatically generate material management business (including material information, processing method, handler, processing deadline, etc.) and associate it with corresponding feedback.

[0082] S4: Perform business processing based on the material management business information, and then feed back the business processing results to the on-track end, and then feed back to the corresponding staff on the on-track end.

[0083] In this step, the business that needs manual processing is manually processed and submitted after the processing is completed.

[0084] In one example, if Figure 8 As shown, after obtaining the material result information and material operation information, the auxiliary marking processing step is also included:

[0085] Correct the material result information and material operation information, and mark reminders according to the urgency of the processing. Specifically, view text, pictures, and voice, edit and correct errors in the automatically recognized and analyzed content, and mark key information (such as operation information, etc.) with labels, and configure colors according to the urgency.

[0086] In one example, if Figure 8 As shown, before business processing and updating based on material management business information, the feedback distribution step is also included:

[0087] The material management business information is distributed to the corresponding users who perform material operations for processing. Specifically, the business that needs to be processed is distributed to different processors according to the processors, and messages are generated to remind the processors to process; for feedback information that has not been processed within the time limit, the processor is automatically reminded with a warning color.

[0088] In one example, if Figure 8 As shown in the figure, after business processing is performed according to the material management business information, the following steps are also included to associate the feedback processing results:

[0089] Corresponding materials are updated according to the business processing results. Specifically, the material business processing results are automatically decomposed and associated with the corresponding feedback.

[0090] In one example, if Figure 8 As shown, after the corresponding materials are updated, the step of confirming the processing results is also included:

[0091] Confirm the business processing results and material update results. Specifically, the ground-based full-time material management personnel will confirm the processing results, and the confirmed results will be sent to the orbit, and the unconfirmed results will be sent back for reprocessing.

[0092] The above specific implementation methods are detailed descriptions of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A space station material management system based on multi-dimensional feedback information, characterized in that: It includes an on-orbit end and a ground end; the on-orbit end includes a multi-dimensional material information acquisition module, a material operation instruction acquisition module and an on-orbit material management and computing node, the multi-dimensional material information acquisition module includes any combination of a first video and image acquisition device, a first audio acquisition device, a first text acquisition device, a label acquisition device, and a two-dimensional code acquisition device, and is used to collect material information, and the material is provided with a label and / or a two-dimensional code; the material operation instruction acquisition module is any combination of a second video and image acquisition device, a second audio acquisition device, and a second text acquisition device, and is used to collect material operation instructions; The on-orbit material management computing node is used to solve material information and material operation instructions to obtain material result information and material operation information. The material result information includes material name, material location information, material shape and type, material number, and material status information; The ground end includes a distributed operation center, which includes a ground material management computing node, which includes a material business processing module and a feedback business processing module. The material business processing module is used to classify and merge materials and material operations according to material result information and material operation information, and generate material management business information, including the operation method of each material, the user who performs the material operation, and the processing deadline; The feedback business processing module is used to perform business processing based on the material management business information, and then feed back the business processing results to the on-track end.

2. The space station material management system based on multi-dimensional feedback information according to claim 1 is characterized in that: The on-orbit end also includes a mobile material management device, on which a mobile material management system is integrated, and on-orbit material management computing nodes are integrated with an on-orbit material information management system.

3. The space station material management system based on multi-dimensional feedback information according to claim 1, characterized in that: The on-track end also includes an access control system for managing the entry and exit and movement records of materials.

4. The space station material management system based on multi-dimensional feedback information according to claim 1, characterized in that: The ground material management computing node is integrated with a ground material information management system.

5. The space station material management system based on multi-dimensional feedback information according to claim 1 is characterized in that: The ground material management and computing node also includes a data solving module, which is used to receive and solve material information and material operation instructions when the on-orbit material management and computing node is unable to solve the data results.

6. A space station material management method based on multi-dimensional feedback information, characterized in that: The method comprises the following steps: Multi-dimensional material information collection and material operation instructions; the multi-dimensional material information collection module includes any combination of a first video and image collection device, a first audio collection device, a first text collection device, a label collection device, and a QR code collection device; Solve the material information and material operation instructions to obtain material result information and material operation information. The material result information includes material name, material location information, material shape and type, material number, and material status information; Classify and merge materials and material operations according to material result information and material operation information to generate material management business information, including the operation method of each material, the user who performs the material operation, and the processing deadline; Business processing is carried out according to the material management business information, and then the business processing results are fed back to the on-track end.

7. The space station material management method based on multi-dimensional feedback information according to claim 6 is characterized in that: After obtaining the material result information and material operation information, it also includes: Correct the material result information and material operation information, and mark and remind them according to the urgency of the processing.

8. The space station material management method based on multi-dimensional feedback information according to claim 6 is characterized in that: Before business processing and updating based on material management business information, it also includes: Distribute material management business information to users who are responsible for performing material operations for processing.

9. The space station material management method based on multi-dimensional feedback information according to claim 6 is characterized in that: After business processing based on material management business information, it also includes: Update corresponding materials based on business processing results.

10. The space station material management method based on multi-dimensional feedback information according to claim 9 is characterized in that: After the corresponding materials are updated, it also includes: Confirm the results of business processing and material update.

Images