Portable intelligent office assistance device

Abstract

The present disclosure provides a portable intelligent office assistance device, comprising: a scanning apparatus, provided with an image sensor, and capable of scanning a surrounding environment and extracting facial feature information and item feature information; an intelligent information processing apparatus, which receives the facial feature information and the item feature information from the scanning apparatus, and comprises a facial recognition module, a digital key module and an item management module, wherein the facial recognition module performs facial recognition on the basis of the facial feature information and determines an authorized user on the basis of the result of the facial recognition, the digital key module acquires, on the basis of the result of the facial recognition, authorization that enables the authorized user to access another office device, and the item management module performs information summarization processing on the item feature information, and according to item locating prompt information provided by the authorized user, infers an item locating result on the basis of the item feature information on which information summarization processing has been performed; and a display apparatus, capable of displaying an output of the intelligent information processing apparatus.

Description

Portable smart office auxiliary devices Technical Field

[0001] This disclosure relates to intelligent office auxiliary devices, and in particular, to portable multifunctional intelligent office auxiliary devices that can improve access security and enable the management of office supplies. Background Technology

[0002] With the rapid development of artificial intelligence technology, traditional office terminal devices such as mobile phones, computers, and laptops have made significant progress in terms of intelligence. For example, current intelligent office assistance devices can conveniently provide us with various office assistance functions, enabling interaction between the network and the physical world. In addition, most intelligent office assistance devices now use fingerprint recognition or facial recognition technology, thereby ensuring user privacy and security.

[0003] However, as shown in Figure 1, existing smart office assistance devices still suffer from the following four problems. First, as shown on the right side of Figure 1, traditional smart office assistance devices excel at information management but are weak in item management. For example, for non-intelligent items such as laptops, glasses, and pens, existing technologies like Apple Tag are not suitable for large-scale item tracking and management. This results in complex management devices and high costs associated with large-scale operations. Second, as shown on the bottom side of Figure 1, most current smart office assistance devices often struggle to balance privacy and access security. For example, to ensure a high level of privacy and security, most smart applications currently use facial recognition technology. However, this also means that users' facial information is at risk of being accessed by various application backends. Third, as shown on the left side of Figure 1, current smart office assistance devices often cannot achieve cross-platform integration across PCs, mobile phones, and tablets. Nearly 80% of applications across these three platforms may be forced to expand their platform or are redundant. This often places a triple burden on users in terms of physical, financial, and mental resources. Finally, as shown in Figure 1, point 4, existing smart office auxiliary devices often organize and provide applications around a specific function, and guide users using big data push notifications. This often overwhelms users with excessive information, preventing them from truly experiencing the efficiency and convenience that smart products should bring. Summary of the Invention

[0004] The technical problem that the invention aims to solve

[0005] In view of the above problems, this disclosure aims to provide a portable, multi-functional intelligent office auxiliary device that can improve access security and enable the management of office supplies.

[0006] Technical means to solve technical problems

[0007] According to embodiments of this disclosure, a portable intelligent office auxiliary device is provided. The intelligent office auxiliary device includes: a scanning device equipped with an image sensor capable of scanning the surrounding environment and extracting facial feature information of people and object feature information of objects in the surrounding environment; an intelligent information processing device receiving the facial feature information and object feature information from the scanning device, and including a facial recognition module, a digital key module, and an object management module; and a display device capable of displaying the output of the intelligent information processing device. The facial recognition module performs facial recognition based on the facial feature information and determines an authorized user existing in the surrounding environment based on the result of the facial recognition; the digital key module obtains authorization for the authorized user to use the portable intelligent office auxiliary device to access another office device based on the result of the facial recognition; the object management module performs information summarization processing on the object feature information and infers the object search result based on the object search prompt information provided by the authorized user and the processed object feature information.

[0008] In a portable intelligent office auxiliary device according to an embodiment of the present disclosure, the image sensor may include an image acquisition unit and a feature extraction unit. The image acquisition unit acquires image information of the surrounding environment, and the feature extraction unit extracts facial feature information and object feature information based on the image information. Preferably, the image sensor is an intelligent image sensor that extracts only the facial feature information and object feature information from the surrounding environment. Preferably, the object feature information includes at least one of the following: object position, color distribution frequency, texture roughness, area, shape, and angular features.

[0009] In a portable intelligent office auxiliary device according to an embodiment of the present disclosure, the digital key module can access the other office device by using WebAssembly technology based on the result of facial recognition to obtain access authorization for hardware and software facilities at a specified level.

[0010] In a portable intelligent office auxiliary device according to an embodiment of this disclosure, the information summarization processing includes at least: a key item determination process for identifying key items among items; and an information compression and storage process for compressing and storing the item feature information of items in the surrounding environment based on spatial dimensions. In the key item determination process, the item management module can determine surrounding auxiliary items existing around the authorized user based on the face recognition result of the face recognition module, and determine key items among the surrounding auxiliary items based on the item feature information of the surrounding auxiliary items. Preferably, the item management module can confirm the surrounding auxiliary items of the authorized user based on the contact frequency between the authorized user and items existing in the surrounding environment. Preferably, the item management module of the intelligent information processing device determines the key item based on a key item feature value calculated based on the item feature information of the surrounding auxiliary items.

[0011] In a portable intelligent office auxiliary device according to an embodiment of the present disclosure, the item management module can use a lightweight large language model running on the intelligent information processing device to perform information compression and storage processing according to the hierarchical dependency relationship between the key item and other items. In the information compression and storage processing, the item management module can extract the hierarchical dependency relationship between the key item and other items based on the spatial dimension-based correlation, and construct a map linked list for compressing and storing the item feature information based on the hierarchical dependency relationship. Preferably, the item management module filters a portion of the item feature information based on feature saliency to determine the compressed feature information in the item feature information of other items, and stores the compressed feature information in the map linked list. In the map linked list, preferably, the item feature information associated with one item corresponds to one storage node, and the reference time is used as the head of the map linked list.

[0012] In the portable intelligent office auxiliary device according to embodiments of this disclosure, the information summarization processing further includes information update processing to update the compressed and stored item feature information. The item management module performs the information update processing when peripheral items existing around the authorized user change or disappear, or when new peripheral items appear around the authorized user. Preferably, when the features of items other than the key item among the peripheral items change or disappear, or when new peripheral items appear, the item management module performs the information update processing to update the compressed feature information stored in the map linked list, and records the time of these changes relative to the reference time. Preferably, when the features of the key item in the scene change, or when a new peripheral item is identified as the new key item, the item management module performs the information update processing to update the compressed feature information stored in the map linked list, and stores the change time in the head of the map linked list as the new reference time.

[0013] In a portable intelligent office auxiliary device according to an embodiment of the present disclosure, the item management module adds the item search prompt information to the input of a micro-generative model running on the intelligent information processing device, expands the compressed feature information stored in the map linked list, and thereby obtains the item search result.

[0014] The portable intelligent office auxiliary device according to the embodiments of this disclosure may further include an input device capable of inputting the item search prompt information, and the display device displays the item search result to the authorized user. The authorized user inputs correction prompt information through the input device, and the item management module obtains the corrected item search result based on the current item search result and the correction prompt information.

[0015] In a portable intelligent office assistance device according to an embodiment of the present disclosure, the intelligent information processing device further includes a joint motion recognition module and an input operation capture module, and the image sensor is also capable of extracting joint information of a person in the surrounding environment. Preferably, the joint information is the joint information of the finger joints of a person in the surrounding environment. The joint motion recognition module can perform motion recognition on the joint movements of the authorized user based on the joint information from the image sensor and the face recognition result from the face recognition module, and, upon recognizing a valid movement of the authorized user, cause the intelligent office assistance device to enter an input operation capture mode. In the input operation capture mode, the input operation capture module of the intelligent office assistance device can generate simulated motion information of a simulated movement that matches the valid movement. Preferably, in the input operation capture mode, the intelligent information processing device controls the display device to display a simulated keyboard and / or simulated touchpad for input to the other office device, and obtains simulated action information of the simulated action performed on the simulated keyboard and / or simulated touchpad based on the keypoint information by using a motion prediction model running on the intelligent information processing device, and uses the simulated action information to control input to the other office device and control the display device to display the simulated action.

[0016] The portable intelligent office auxiliary device according to the embodiments of this disclosure may further include a voice input device, through which the authorized user inputs voice control information, and the intelligent information processing device parses the voice control information and combines the parsed voice control information with the simulated action information to control input to the other office device and control the display device to display the simulated action.

[0017] The portable intelligent office assistance device according to embodiments of this disclosure may further include an office assistance module, which grants access to a specified internet address to the portable intelligent office assistance device based on the recognition result of the action recognition by the keypoint action recognition module. Based on the prompt information input by the authorized user using the input device and / or the voice control information input using the voice input device, the office assistance module can automatically generate office support documents using a lightweight large language model and a micro-generative model running on the intelligent information processing device. Preferably, the specified internet address may be a specific enterprise RAG knowledge base and / or a specified website. Beneficial effects

[0018] The portable intelligent office auxiliary device disclosed herein can significantly expand the application scope of facial recognition security functions while effectively ensuring user privacy and security, enabling unified management of hardware and software settings and reducing management costs.

[0019] Furthermore, the portable smart office auxiliary device disclosed herein can help users manage their belongings and solve the problem of forgetting or losing items in daily life.

[0020] In addition, the portable intelligent office auxiliary device disclosed herein can meet the various functional needs of both office and daily scenarios, saving user costs; at the same time, it can provide service-oriented intelligent hardware, improving daily office productivity. Attached Figure Description

[0021] Figure 1 is an illustrative diagram showing the prior art.

[0022] Figure 2 illustrates a schematic diagram of the functionality of a portable smart office aid device according to an embodiment of the present disclosure.

[0023] Figure 3 shows a block diagram of the overall construction of a portable intelligent office assistance device according to an embodiment of the present disclosure.

[0024] Figure 4 is a schematic diagram illustrating the structure and modification method of the map linked list for information compression storage processing of the item management module of a portable intelligent office auxiliary device according to an embodiment of the present disclosure.

[0025] Figure 5 is a schematic flowchart illustrating the overall operation of a portable intelligent office assistance device according to an embodiment of the present disclosure.

[0026] Figure 6 is a flowchart illustrating the item search and matching process of a portable smart office auxiliary device according to an embodiment of the present disclosure.

[0027] Figure 7 is a functional schematic diagram illustrating the input operation capture mode of a portable smart office auxiliary device according to an embodiment of the present disclosure.

[0028] Figure 8 is a schematic diagram illustrating the updated map linked list in the information compression storage processing of the item management module of a portable smart office auxiliary device according to an embodiment of the present disclosure. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] It should be noted that the use of terms such as "approximately" and "about" in the following detailed description and claims is to take into account factors such as manufacturing tolerances and machining accuracy that are understood by those skilled in the art, and will not lead to ambiguity in the description or unclear scope of protection. Furthermore, the orientations such as "upper," "lower," "inner," and "outer" appearing herein are merely for ease of explanation and are defined in conjunction with the directions shown in the accompanying drawings or the common usage or operating states of the device. Those skilled in the art, after reading this document, will be able to easily determine the orientation corresponding to the orientation described herein even when the device exhibits an orientation inconsistent with that described herein due to movement or rotation.

[0031] [Overall Structure]

[0032] The overall structure of the portable intelligent office assistive device according to the present disclosure will now be described with reference to Figures 2 and 3. Figure 2 shows a schematic diagram of the function of the portable intelligent office assistive device according to the technology of the present disclosure, and Figure 3 shows a block diagram of the overall structure of the portable intelligent office assistive device according to an embodiment of the present disclosure.

[0033] The portable intelligent office assistance device 100 according to embodiments of the present disclosure can be deployed in any private or public place where there may be office needs, such as a conference room, office, study, or warehouse, according to user needs. As shown in FIG3, the portable intelligent office assistance device 100 according to embodiments of the present disclosure may include at least a scanning device 110, an intelligent information processing device 120, and a display device 130. The scanning device 110 is capable of scanning and capturing images of the surrounding environment to obtain various information such as image information and distance information required for subsequent processing. For example, the scanning device 110 may be equipped with an image sensor 111, which scans the surrounding environment and is capable of extracting facial feature information of people in the surrounding environment and object feature information of surrounding objects. For example, the image sensor 111 may be a Sony IMX500 intelligent image sensor. Specifically, the image sensor 111 may be equipped with an image acquisition unit 1111 and a feature extraction unit 1112. The image acquisition unit 1111 acquires image information of the surrounding environment, and the feature extraction unit 1112 extracts facial feature information and object feature information based on the image information. Facial feature information is the facial feature information of a person present in an image extracted from image information. The person present in the image may be an authorized user or an unauthorized user of the portable intelligent office assistance device 100. As will be explained below, the intelligent information processing device can identify authorized and unauthorized users based on facial feature information to ensure the privacy and access security of the portable intelligent office assistance device 100. The intelligent information processing device may be at least one of the information processing devices capable of efficiently performing tasks such as image processing, speech recognition, and natural language processing, such as an artificial intelligence image information processor (AI ISP), a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a DSP (digital signal processor), and a field-programmable gate array (FPGA). The display device 130 may be, for example, a projection device, a monitor, a display screen of a personal mobile device, or various other known display devices.

[0034] Item feature information is the feature information of items existing in an image extracted from image information. In this technology, the most important items in the image are those existing around the authorized user (hereinafter also referred to as surrounding accessories). Item feature information may include one or more of the following features: item location, color distribution frequency, shape and area, texture roughness, etc. For example, the image sensor 111 can extract feature vectors using traditional image algorithms to obtain common feature information of items, especially surrounding accessories, such as color distribution frequency, shape and area, texture roughness, and classify and locate surrounding accessories existing in the environment around the authorized user (hereinafter also referred to as the surrounding environment).

[0035] According to the technology of this disclosure, the image sensor 111 of the scanning device 110 can use any known suitable item recognition and classification technology to classify items in images captured by the scanning device 110. For example, according to one embodiment of this disclosure, the image sensor 111 can use a backbone network to classify items in the image to obtain item feature information including item name and item location information (e.g., 10-bit item classification information). For example, the image sensor 111 can infer the location information of the item's bounding box (e.g., the four coordinate points of the item's bounding box (4 bytes * 8)). In addition, feature vectors can be extracted for faces or unclassifiable items. For example, facial vectors can be used as feature vectors for faces, and REID (person re-identification) information can be used as feature vectors for unclassifiable items. The image sensor 111 of the scanning device 110 sends the item feature information, including item classification and location information, to the intelligent information processing device 120. The intelligent information processing device 120 processes the received data through parsing, authentication, and storage. It then compares the data with relevant information in a database stored in its memory, updating the item's status and location (using four coordinate points, e.g., represented by 4 bytes * 8). Furthermore, by deploying multiple scanning devices 110 at various locations, the readers in these devices automatically read the tag information and upload it to the intelligent information processing device 120 when an item passes nearby, enabling real-time tracking of the item. This allows for real-time monitoring of the item's location and movement. The intelligent information processing device 120 also provides various query and analysis functions, allowing for real-time viewing of the item's location, status, and historical trajectory. This not only helps managers understand the movement of items but also enables analysis for inventory management and supply chain optimization.

[0036] To ensure user privacy, according to a preferred embodiment of this disclosure, the image sensor 111 of the scanning device 110 can be an intelligent image sensor that extracts only facial feature information of people present in the surrounding environment, without extracting complete facial image information. In other words, in this case, the intelligent information processing device 120 will not receive any complete facial image information, but will only receive facial feature information that has been processed by the image sensor 111 of the scanning device 110. This greatly improves the privacy and security of the portable intelligent office auxiliary device 100.

[0037] According to the present disclosure, the intelligent information processing device 120 of the portable intelligent office auxiliary device 100 receives facial feature information and object feature information from the scanning device 110, and may include, for example, a facial recognition module 121, a digital key module 122, and an object management module 123. The facial recognition module 121 can perform facial recognition based on the facial feature information from the scanning device 110, and determine the authorized user present in the surrounding environment based on the facial recognition result. Once determined to be an authorized user, the portable intelligent office auxiliary device 100 can be unlocked by the authorized user for subsequent operations. Otherwise, the portable intelligent office auxiliary device 100 will remain in standby mode. The digital key module 122 can, based on the facial recognition result, enable the authorized user to use the portable intelligent office auxiliary device 100 to access and obtain authorization for use of another office device such as a server, PC, laptop, projector, or printer via wired or wireless communication.

[0038] Specifically, the digital key module 122 uses WebAssembly technology to access another office device based on facial recognition results to obtain access authorization for a specified level of hardware and software facilities. The WebAssembly program in the digital key module 122 runs in a sandbox environment, the key pair is generated through a Hardware Security Module (HSM), and the transmission process is encrypted using encryption protocols such as TLS 1.3. For example, the portable smart office auxiliary device 100 sends the RSA public key WebAssembly program containing facial information to a receiving device that also supports Wi-Fi or Bluetooth protocols via its communication module. The receiving device first runs a WebAssembly verification program to ensure that the RSA public key WebAssembly program has not been contaminated (missing data) or maliciously attacked during transmission. Then, the RSA public key WebAssembly program, combined with the RSA private key file sent by the receiving device's WebAssembly program, performs data decoding. Upon successful decoding, the receiving device's WebAssembly program continues to match the received facial feature information with facial feature information stored in a facial information database. Upon successful matching, the receiving device sends a matching success signal (match success flag and authorization level) from the WebAssembly sandbox environment to the portable intelligent office assistance device 100, thereby granting the portable intelligent office assistance device 100 the corresponding authorization. If an error occurs in any of the above steps, decoding or matching will fail, and the portable intelligent office assistance device 100 will need to resend the RSA public key WebAssembly procedure or relinquish authorization.

[0039] According to the item management module 123 of the intelligent information processing device 120 of the portable intelligent office auxiliary device 100 of this disclosure, based on the face recognition result of the face recognition module 121, the item feature information of the items around the authorized user is summarized and processed, including key item identification processing and information compression and storage processing. For example, in the standby monitoring mode (also known as idle state) of the portable intelligent office auxiliary device 100, the image sensor 111 of the scanning device 110 automatically scans the surrounding environment and confirms the surrounding accessories of the authorized user based on the face recognition result, filtering out the unauthorized user and their surrounding environment, thereby improving the identification efficiency of the surrounding accessories of the authorized user while protecting the privacy and security of the unauthorized user. Specifically, the item management module 123 of the intelligent information processing device 120 of the portable intelligent office auxiliary device 100 first confirms the surrounding accessories. For example, it can confirm which items are surrounding accessories based on the frequency of contact between the authorized user and the items in the surrounding environment. Specifically, a contact threshold can be set (e.g., a specific number of contacts within a certain time period). When an authorized user contacts an item more than the threshold within a predetermined time period, the item is considered associated with the authorized user, meaning it is an accessory item belonging to the authorized user. To achieve convenient and intelligent management of accessory items, the item management module 123 of the intelligent information processing device 120 then extracts features from the accessory items' characteristic information and determines key items among the accessory items based on the key item feature values ​​calculated from the extracted feature information. After determining the key items, the layer-by-layer dependency relationships between other items and the key items are extracted based on the spatial dimension correlation, thereby achieving spatial dimension-based information compression and storage (i.e., information compression and storage processing). In addition, the intelligent information processing device 120 can also update the stored compressed information based on the time dimension.

[0040] First, the identification of key items can be based on, for example, the following formula (1): key_object_value=-α1*distance+α2*color_frequency+α3*area_size+α4*coarseness ...... Formula (1)

[0041] Wherein, α1 is the coefficient representing distance weight, distance represents the distance between the item and the authorized user, α2 is the coefficient representing color distribution frequency weight, color_frequency represents the color distribution frequency of the item, α3 is the coefficient representing area weight, area_size represents the area of ​​the item, α4 is the coefficient representing texture weight, and coarseness represents the roughness of the item. Here, α2>α3>α1>α4 can be set. The key object feature values ​​(key_object_value) of the detected surrounding objects are sorted. The surrounding object with the largest key object feature value among all surrounding objects is identified as the key object. It should be understood that additional terms can be added to the above formula (1) according to the actual situation. For example, in a dimly lit place, a term consisting of a coefficient representing brightness weight and the brightness of the item can be added. Furthermore, the magnitude of each coefficient can be adjusted according to actual needs. Alternatively, other formulas can be selected or designed as needed to calculate the key object feature value, as long as the calculation result of the formula can highlight the importance of each item in the environment relative to the user.

[0042] After key items are identified, the item management module 123 then uses a lightweight large language model running on the intelligent information processing device 120 to perform information compression and storage processing based on spatial dimensions to compress and store the item feature information of surrounding auxiliary items. In summary, this information compression processing roughly includes feature vector dimensionality reduction, spatial correlation analysis and extraction, and the removal of redundant information. This information compression and storage processing can greatly reduce the amount of information that needs to be processed and stored when implementing item management functions, significantly reducing the cost required to implement item management and search functions and improving work efficiency. Here, the lightweight large language model refers to a common neural network model based on the Transformer architecture, which, through pruning and quantization techniques, can compress the number of parameters to less than 20% of the original size for natural language inference and feature correlation analysis. This information compression and storage processing is mainly achieved by constructing a map linked list for compressing and storing item feature information.

[0043] Figure 4 illustrates the structure of the map linked list constructed by the item management module 123 during the information compression and storage process. As shown in Figure 4, to compress and store the item feature information of items in the scene, the item management module 123 establishes a spatial dimension-based map linked list of information for all items in the scene, centered on key items and arranged in a certain order (e.g., from closest to furthest radius, clockwise). Each item corresponds to a node in the linked list. The key of each node is the most prominent item feature information of the corresponding item and the name of the identified item. If an item cannot be identified, the geometric mean of the REID coordinate points is used as the feature, and the item name can be uniformly defined as such as Unknown Item 1, Unknown Item 2, etc. Here, the selection and determination of the most prominent item feature information of an item is a process of filtering and eliminating a portion of the item's feature information based on feature saliency. For example, the numerical values ​​of item feature information that can be quantified, such as distance from the key item, color distribution frequency, area, and roughness, are sorted for all items except the key item. Then, the item feature with the highest numerical ranking among all item feature information is taken as the most significant item feature, and other item features with lower rankings are removed. For example, if an item's distance from the authorized user ranks 5th, color distribution frequency ranks 2nd, area ranks 3rd, and surface roughness ranks 9th among all items, then the item feature with the highest significance is considered to be color distribution frequency. Only the item's color distribution frequency is stored in the corresponding storage location of the map linked list, while item features related to the item's distance from the authorized user, area, and surface roughness are removed. In the information storage structure of the map linked list, the value of an item's node includes information about related items in the four directions of the item: front, back, left, and right. For example, 4 directions * (2 directional bits (representing front, back, left, and right) + 10 item category information (supporting more than 1000 common easily lost items)) totaling 48 bits (6 bytes) of information. This 48-bit information can contain information about all items in the four directions associated with a certain item, thereby realizing the extraction and expression of the hierarchical dependency relationship of items based on the spatial dimension, which greatly reduces the amount of information that needs to be stored and processed for item management.

[0044] Furthermore, the item management module 123 can also perform information updates based on the time dimension. Figure 8 illustrates a schematic diagram of the updated map list in the item management module 123 of the portable intelligent office auxiliary device according to an embodiment of the present disclosure. In the above map list, the head is the base time. The initial base time is the time when the item management module 123 first determines the key item. When the scanning device 110 detects that the characteristics of other items in the scene besides the key item have changed, disappeared directly, or new items have appeared, the key item confirmation and spatial dimension-based information compression processing described above will be re-executed, but the base time will not be updated. Only the latest relative change time value will be written to the time feature area of ​​the node of the changed item (if it is the same as the base time, it will not be stored). When the characteristics of the key item in the scene change, or when other items change and are reconfirmed as new key items, in addition to re-executing the key item confirmation and spatial dimension-based information compression processing described above, the time of the change will also be used as the new base time.

[0045] Through the aforementioned information processing, the item management module 123 of the intelligent information processing device 120 can quickly provide item search results based on the processed item feature information, according to the item search prompts provided by the authorized user. In this process, a lightweight large language model is responsible for semantic understanding and contextual reasoning (such as parsing the user's item search prompt "Where are my keys?"). The intelligent information processing device 120 can then send the search results to the display device 130 and display them on the display device 130.

[0046] During the item search process, the item management module 123 can also perform an approximate search based on the item search prompts and compressed features of surrounding items, and use a micro-generative model running on the intelligent information processing device to expand the information to obtain preliminary item prediction information. The micro-generative model here is a commonly used generative network based on a single-layer Transformer decoder, which may contain 128 hidden layers and 4 attention heads, with 0.5M parameters. In this application, the micro-generative model is used to convert compressed and stored item feature vectors into natural language descriptions. In other words, the micro-generative model is responsible for generating specific outputs from the feature vectors. For example, when a user queries "glasses location," the micro-generative model reads the feature vector (color, relative position) of the glasses node from the map linked list and generates the output "The glasses are located on the left side of the computer, and the last scan time was 10:00 AM." The portable intelligent office auxiliary device 100 may also include an information input device 141 capable of inputting item search prompts, and the display device 130 displays the preliminary item prediction information to the authorized user, who can input correction prompts through the information input device 141 based on the preliminary item prediction information. The item management module 123 then performs another search based on the preliminary item prediction information and correction prompt information to obtain the corrected item search results. This will be explained in detail later. It should be noted that the information input device 141 mentioned here, which can input item search prompt information, and the voice input device 142 mentioned below, etc., can be collectively referred to as the "input device 140" of the portable intelligent office auxiliary device 100.

[0047] The portable intelligent office assistance device 100 according to a preferred embodiment of this disclosure can also greatly improve the user's office efficiency and convenience. For example, the image sensor 111 of the scanning device 110 can also extract the joint information of the authorized user in the surrounding environment, and the intelligent information processing device 120 further includes a joint motion recognition module 124 and an input operation capture module 125. For example, the joint information here may be the joint information of the user's finger joints in the surrounding environment. The joint motion recognition module 124 performs motion recognition on the joint motion of the authorized user based on the joint information from the image sensor 111 and the face recognition result from the face recognition module 121 of the intelligent information processing device 120, and, when a valid motion of the authorized user is recognized, causes the portable intelligent office assistance device 100 to enter the input operation capture mode that generates simulated motion information. The simulated motion here refers to the machine virtual motion that is recognized and generated by the intelligent information processing device 120 and matches the valid motion of the authorized user's joints.

[0048] In input capture mode, the portable intelligent office assistant device 100 can control the display device 130 to display the simulated keyboard and / or simulated touchpad for input from the connected office equipment. The input capture module 125, using a motion prediction model running on the intelligent information processing device 120, obtains simulated motion information based on keypoint information, matching the valid actions of the authorized user to the simulated keyboard and / or simulated touchpad. It then uses this simulated motion information to control input to the connected office equipment and controls the display device 130 to display the simulated actions generated by the intelligent information processing device 120. In this case, the portable intelligent office assistant device 100 may also include a voice input device 142. The authorized user inputs voice control information through the voice input device 142. The intelligent information processing device 120 of the portable intelligent office assistant device 100 can parse the voice control information and combine it with the simulated motion information to control input to the connected office equipment and control the display device 130 to display the simulated actions generated by the intelligent information processing device 120. For example, voice control can be used to open and close software, switch pages, etc. This combination of voice analysis and motion capture is faster than traditional mouse control.

[0049] According to a preferred embodiment of the portable intelligent office assistance device 100 of this disclosure, the intelligent information processing device 120 may further include an office assistance module 126 with customized office assistance functions. For example, the office assistance module 126 may grant access to a specified Internet address to the portable intelligent office assistance device 100 based on the results of motion capture recognition, and automatically generate office support documents using a lightweight large language model and a micro-generation model running on the intelligent information processing device 120, based on prompts input by the authorized user using the information input device 141 or voice control information input using the voice input device 142. The specified Internet address here may be, for example, a specific enterprise RAG knowledge base and / or a specified website. RAG is a conventional workflow technology, which can be understood here as a Python program module that extracts PPT styles. The user inputs content as prompts, and RAG combines the prompts and the title of the PPT template to search for similar content in the database. Subsequently, the micro-generation model summarizes the relevant content and fills the relevant areas with the summarized content and related explanatory images according to the style of the PPT template to complete the entire PPT generation process. For example, authorized users can first upload their own PPT templates, and then the intelligent information processing device 120 uses a large language model to perform similarity searches and summaries on the connected RAG devices based on the user's input, and generates corresponding PPT documents according to the template style.

[0050] [Overall Equipment Operation Procedure]

[0051] The overall operation flow of the portable intelligent office assistance device 100 according to an embodiment of the present disclosure will now be described with reference to Figures 2, 5, and 6. For ease of explanation, the different modes of the overall operation flow will be divided into four states for description. However, it should be understood that, apart from the initialization mode and standby monitoring mode described below, the order of the other states in the following text does not mean that these states occur in chronological order, but rather that they can occur in any order or in parallel according to the customer's operation.

[0052] First state – Initialization mode (the upper left box of Figure 2 and Figure 5):

[0053] First, the portable smart office assistance device 100 is deployed in the required office locations, and its initial settings are performed. The user of the portable smart office assistance device 100 then registers their facial information. As mentioned earlier, during the facial information registration process, the image sensor 111 of the scanning device 110 can only register the user's facial feature information, without registering the user's complete facial image, thereby ensuring the user's privacy and security.

[0054] Second state – Standby monitoring mode (top right box of Figure 2, Figures 5 and 6):

[0055] After initialization, the portable intelligent office auxiliary device 100 automatically enters standby monitoring mode. As mentioned above, at this time, the scanning device 110 of the portable intelligent office auxiliary device 100 scans the objects in the surrounding environment of the authorized user whose face verification is matched, in order to obtain the facial feature information and object feature information of the surrounding environment. The scanning device 110 sends this information to the intelligent information processing device 120, which performs information summarization processing as described above. First, the intelligent information processing device 120 confirms the surrounding objects of the authorized user and extracts the object feature information (such as color, area, size, etc.) of all surrounding objects. Then, the intelligent information processing device 120 uses formula (1) to calculate the key object feature value and determines the key object among the surrounding objects. In this way, the intelligent information processing device 120 completes the key object determination process. Next, the intelligent information processing device 120 takes the key object as the center and extracts the corresponding feature information of other surrounding objects according to the correlation and dependence of other surrounding objects with the key object as described above. Subsequently, a lightweight large language model running cyclically on the intelligent information processing device 120 summarizes the item feature information of surrounding objects at a specific moment, thereby constructing and updating the map linked list mentioned above as shown in Figure 4 in a timely manner. The intelligent information processing device 120 will continue to perform the processing in this stage until the portable intelligent office auxiliary device 100 actively or passively switches to other modes.

[0056] In standby monitoring mode, when an authorized user needs to find an item, they can input search prompts (also known as prompt information) via voice or text through input devices 140 such as a miniature microphone or keyboard on the portable intelligent office auxiliary device 100 (e.g., on a certain day, this item was seen, but it cannot be found now; where is the item currently located, etc.). Upon receiving the prompt information, the intelligent information processing device 120 runs the miniature generation model stored therein, combining the prompt information with item information in the constructed map list. First, the miniature generation model generates expanded prompt information. Then, using the expanded prompt information as input, it generates preliminary search results containing location and item feature information, and feeds these search results back to the authorized user through a display device 130 (e.g., a miniature projector). If the search result matches the authorized user's needs (e.g., the category and location of the item as remembered by the authorized user), the search process is completed, and the portable intelligent office assistance device 100 automatically exits the item search process and returns to the standby monitoring mode; otherwise, the user can further submit a revised prompt, and the intelligent information processing device 120, in conjunction with the currently given search result, repeats the above generation process until the search result inferred by the device matches the user's needs.

[0057] Third state – Input operation capture mode (lower left box of Figure 2, Figure 5 and Figure 7):

[0058] During the standby monitoring phase, when the intelligent information processing device 120 of the portable intelligent office assistance device 100 identifies an authorized user in the image scanned by the scanning device 110, and identifies the authorized user's hand movements as certain specially defined actions (such as keyboard typing or text input actions), the portable intelligent office assistance device 100 will enter the input operation capture mode to capture the authorized user's hand movements, so as to achieve faster office assistance functions.

[0059] For example, when the joint motion recognition module 124 of the intelligent information processing device 120 recognizes the effective movement of the authorized user's hand joints based on the joint information from the image sensor 111 of the scanning device 110, the portable intelligent office assistance device 100 controls its miniature projection device to project a simulated keyboard and / or simulated touchpad onto a desktop, wall, or projection screen. Furthermore, the intelligent information processing device 120, by running a stored lightweight large language model for motion prediction, obtains simulated movements based on the joint information that match the operator's effective hand movements on the desktop, and uses the simulated movement information of the simulated movements to achieve accurate input on the simulated keyboard and / or simulated touchpad. Additionally, as described above, voice input can also be used; the lightweight large language model running on the intelligent information processing device 120 parses the voice input and combines it with simulated keyboard input to achieve efficient input control of applications on the access device. For example, voice is responsible for macro-level input commands (e.g., opening an application interface), while hand input is responsible for fine-tuning the input content.

[0060] Fourth state – Office auxiliary operation mode (bottom right box of Figure 2 and Figure 5)

[0061] According to a preferred embodiment of this disclosure, the portable intelligent office assistance device 100 can also enter an office assistance operation mode based on instructions from an authorized user. When the portable intelligent office assistance device 100 enters the office assistance operation mode, it can realize a variety of specific office assistance functions. For example, as shown in FIG2, when an authorized user creates a document using the portable intelligent office assistance device 100, if the intelligent office assistance device 100 is granted access to the enterprise RAG knowledge base and / or internet access, then through the lightweight large language model running on the intelligent information processing device 120, the intelligent information processing device 120 can automatically generate supporting charts based on the information returned from the RAG knowledge base and / or internet database and display them to the authorized user, thereby accelerating the entire document completion process.

[0062] Finally, after completing a specific office assistance function, the portable smart office assistance device 100 determines whether it needs to be powered off. For example, if the power button is pressed, the portable smart office assistance device 100 will power off; if it is not pressed, the portable smart office assistance device 100 can re-enter standby monitoring mode.

[0063] [Specific examples of item information association and search]

[0064] As described above, the intelligent information processing device 120 of the portable intelligent office auxiliary device 100 according to this disclosure can summarize and process the item feature information obtained by the image sensor 111, establish the association between items, and store the relevant information in a compressed form, thereby realizing fast and intelligent item search. The following example will specifically illustrate how to achieve compressed storage and item search based on the hierarchical dependency relationship of items by establishing a map linked list.

[0065] For example, suppose an authorized user has glasses, a computer, keys, a coffee cup, a ballpoint pen, and an unidentified item as accessories.

[0066] First, through scanning and processing by the image sensor 111 in the scanning device 110, the intelligent information processing device 120 can obtain, for example, the information of the item listed in Table 1 below. It should be noted that Table 1 only lists the item name, category number, and location information as an example. In reality, the intelligent information processing device 120 can also obtain various other information such as the item's area, color, and surface roughness.

[0067] Table 1

[0068] For unidentified objects that cannot be classified temporarily, a 128-dimensional feature vector can be taken (to distinguish different unidentified objects), and then the average value can be calculated as the most prominent feature of the item. An example of a 128-dimensional feature vector is as follows: unknow_id=[0.123,0.456,0.789,0.234,0.567,0.890,0.135,0.246,0.357,0.468,0.579,0.680,0.791,0.802,0.913,0.024,0.135,0.2] 46, 0.357, 0.468, 0.579, 0.680, 0.791, 0.802, 0.913, 0.024, 0.135, 0.246, 0.357, 0.468, 0.579, 0.680, 0.791, 0.802, 0.913, 0.024, 0.135, 0.246, 0.357, 0.468, 0.579, 0.680, 0.791, 0.802, 0.913, 0.024, 0.135, 0.246, 0.357, 0.468, 0.579, 0.680, 0.791, 0.802, 0.913, 0.024, 0.135, 0.246, 0.357, 0.468, 0.579, 0.680, 0.791, 0.802, 0.913, 0.024, 0.1 35,0.246,0.357,0.468,0.579,0.680,0.791,0.802,0.913,0.024,0.135,0.246,0.357,0.468,0.579,0.680,0.791,0.802,0.913,0.024,0.135,0.246,0.357,0.468,0.579,0.680,0.791,0.802,0.913,0.024]

[0069] The scanning device 110 sends the item information acquired by the image sensor 111 to the intelligent information processing device 120. The intelligent information processing device 120 first uses the time information as the head of the map linked list, and then determines the key items. Specifically, it calculates the key item feature value for each item according to Formula 1 above. For example, the computer's area_size = 160 * 210, distance from the authorized user = 30, color_frequency = 90 (all black), and coarseness = 20, therefore the key_object_value is calculated to be 500. Through similar calculations, it is known that the computer has the largest key item feature value among all items, therefore the computer is designated as the key item.

[0070] After identifying the key items, the intelligent information processing device 120 performs spatial-dimensional information compression and storage processing on the item feature information. Assuming 10:00 AM as the base time, and centering on the key item (computer), the relative positions of other auxiliary items are as follows: To the left of the computer are glasses (0b10 represents the left side, glasses are categorized as 30, corresponding to 10 bits of information 0b00,0001,1110) (where 0b represents binary and does not occupy information bits), below are keys (0b01 represents the bottom, keys are categorized as 32, corresponding to 10 bits of information 0b00,0010,0000), to the right are unidentified objects (0b11 represents the right side, unidentified objects are categorized as 1001 as above, corresponding to 10 bits of information 0b11,1110,1001), and above are no objects, with a default value of 0. This allows the construction of a map linked list as shown in Table 2.

[0071] Table 2

[0072] Referring to Figure 4 and Table 2 above, in the map linked list, the first node after the base time is the key item node. Subsequent nodes can be arranged in a predetermined appropriate order. Table 2 does not list the contents of all data nodes in detail. By centering on the key item and, for example, following a clockwise order from nearest to farthest by radius, similar information associations are performed on the item feature information of all items, and then stored. This completes the compression and storage of item feature information extracted based on the hierarchical dependency relationships of items throughout the entire scene.

[0073] When the characteristics of an item in the image change or a new item is added, the item management module 123 of the intelligent information processing device 120 can perform time-based information updates as described above. For example, as shown in Figure 8, at 11:00 AM, the scanning device 110 detects a copper key placed to the left of a coffee cup. Through simple comparison, the item management module 123 detects that the number of items has increased. The intelligent information processing device 120 will re-execute the process of confirming the key item. In this example, the key item is still the computer. Since the key item has not changed, the base time is still 10:00 AM, but the change time information is recorded as +1 hour in the node of the added item "copper key". Then, with the key item computer as the center, the item characteristic information of all other items is spatially compressed according to certain rules (e.g., from near to far in a clockwise direction based on radius distance). In this way, the corresponding characteristic information of the copper key is added to the updated map list.

[0074] When an item disappears from the scene, the distance value in Formula 1 is set to infinity. The item management module 123 of the intelligent information processing device 120 records the relative time of the item's last appearance (i.e., the time change value relative to the base time) in the "Change Time" field of the node, and moves the node storing the item's related information to the last position in the map list. When the content stored in the map list exceeds the storage limit, the content stored at the last position of the map list will be "squeezed out" to the "cold storage" described below. Before storing the item's information in the "cold storage," the intelligent information processing device 120 adds the time information based on the base time and the "Change Time" in the node to restore the real time. In addition, the specific relative position information and characteristics of the item will be deleted, and only the real time of the item's last appearance, the item name, and the broadest position information will be saved (e.g., if the key disappears at 11:00 AM, its specific relative position information and characteristics will be deleted after the update, and only the time information + name and the broadest position information will be retained, such as "am10:00, on the table, there is a key"). Here, "broadest positional information" refers to the positional information that most easily identifies the object's positional relationship with minimal information. For example, it's easy to understand that the relative positional information of an object with respect to the object with the largest area or volume in the scene is typically used as the object's "broadest positional information." For instance, in the previous example, the object with the largest area in the scene is the desk, so the "broadest positional information" of the key is "on the desk." This "broadest positional information" can be obtained by a lightweight large language model by traversing the relative positional information stored in a map linked list.

[0075] The following examples will illustrate the process of finding items.

[0076] For example, an authorized user remembers leaving the key somewhere in the scene, but can't recall the exact location. In this case, several scenarios may occur.

[0077] 1) When searching for a recently recorded item

[0078] i. First, the item management module 123 of the intelligent information processing device 120 traverses the map list in Figure 4 from beginning to end, performing an approximate search on the key values ​​(feature + name) of the item information stored in the map list. At this time, the information of the node in the map list is combined with the reference time to obtain the actual time when the item was last scanned, forming a new key-value pair (key: reference time + change time + feature + name: value (48-bit location information)) and a status flag (True indicates found). This is sent to the lightweight large language model running on the intelligent information processing device 120, which is responsible for converting it into human-understandable language and returning it to the user, for example, prompting the user through voice or on-screen text: "The key is on the desktop, behind the computer screen."

[0079] 2) When the item being searched is an item recorded earlier (for example, the item being searched has not appeared in the scene for a long time, so the location information of the item may no longer be in the map list),

[0080] ii. If the item being searched is in the cold storage or archive storage of the lightweight large language model of the intelligent information processing device 120. The cold storage or archive storage of the lightweight large language model is defined here as data that needs to be stored long-term and is accessed by the lightweight large language model at a low frequency (e.g., once every few weeks or months). In this case, the item management module 123 first traverses the map list in Figure 4 from beginning to end. If no matching key is found in the map list, it searches for it in the summary information stored in the cold storage of the lightweight large language model. When a match is found, the information found in the cold storage (e.g., 10:00 AM on a certain day, on the desktop, there is a key) + the status flag (True) is combined as a new expanded prompt message and input into the micro-generation model. The micro-generation model is responsible for translating this into human-understandable language and returning it to the user, for example, prompting the user via voice or on-screen text: 10:00 AM on a certain day, the key is on the desktop.

[0081] iii. The item being searched for is also not in the cold storage of the lightweight large language model of the intelligent information processing device 120. In this case, the item management module 123 first traverses the map list in Figure 4 from beginning to end. When no matching key is found in the map list, it searches for the summary information stored in the cold storage of the lightweight large language model. If no match is found, the item management module 123 combines all the information stored in the cold storage (e.g., it may include the item feature information of all items in locations such as the office desk, home desk, and dining table) to synthesize new item search prompt information, and at this time, the flag bit is set to false. In this case, the lightweight large language model will deduce the last possible location where the key appeared. The following example illustrates this reasoning and search process.

[0082] a. A lightweight, large language model infers a result that humans can understand: Is the key still on the door?

[0083] The user corrected their reply: I looked for it, it's not on the door, I think it's inside the house.

[0084] b. The item management module 123 inputs the corrected information "I've looked, it's not on the door, I think it's inside the house" and all item feature information stored in the cold storage as new prompt information into the lightweight large language model. The lightweight large language model repeats the reasoning and response synthesis process and responds with the information: What house? Are there many computers in that house?

[0085] The user corrected their reply: I remember there weren't many computers.

[0086] c. The item management module 123 inputs the corrected information "I remember not having many computers" and all item feature information stored in the cold storage as new prompt information into the lightweight large language model. The lightweight large language model repeats the reasoning and response synthesis process and responds with the information: Is it at the dining table at home?

[0087] The user corrected their reply: I've looked, it's not here. It's not usually placed on the dining table either.

[0088] d. The item management module 123 inputs the corrected information "Searched, not here. It's not usually placed on the dining table" along with all item characteristic information stored in the cold storage as new prompt information into the lightweight large language model. The lightweight large language model repeats the reasoning and response synthesis process and responds with the information: "Is it usually placed in the study? Is it on the table?"

[0089] The user corrected their reply: But I didn't find it this time. I remember it was next to the computer and coffee cup before.

[0090] e. The item management module 123 inputs the corrected information "But I didn't find it this time; I remember it being near the computer and coffee cup before" along with all item feature information stored in the cold storage as new prompt information into the lightweight large language model. The lightweight large language model repeats the reasoning and response synthesis process and responds with the information: "Have you looked behind the computer?" It should be noted that after repeated training (for example, after approximately 1000 simulated training tests, the model achieves a 95% accuracy rate in predicting the location of lost items, and the training dataset contains 10,000 historical item location records), the lightweight large language model can determine the location where items are most likely to be lost and preferentially provide the corresponding inference results.

[0091] The user replied: It seems to be there... Found it!

[0092] f. The reasoning and search process ends.

[0093] [Methods and specific examples of hand joint recognition]

[0094] The method and specific examples of hand joint recognition are described in detail below with reference to Figure 7. As shown in Figure 7, the image sensor 111 outputs the obtained position information of the hand joints (upper left side of the figure) and the corresponding optical flow data information (middle left side of the figure) to the intelligent information processing device 120 as input to the multimodal lightweight large language model running on the intelligent information processing device 120. The multimodal lightweight large language model combines the input method selected by the user's voice command and the internal historical text input to infer the user's possible hand movements. As shown in Figure 7, if the effective finger movement has been determined to be typing the letters "Colu", the softmax output can be obtained by combining the hand joint information and optical flow data and the previous training data of the deep learning network large language model. In this case, the role of softmax can be simply understood as "scoring each possible movement and turning these scores into probabilities". As shown in Figure 7, assuming the model wants to predict whether the next action of a hand joint will be "pressing the M key," "pressing the N key," "pressing the J key," or "pressing the H key," it first calculates a "raw score" for each of these four actions (e.g., 3 points for pressing the M key, 2 points for pressing the N key, 1 point for pressing the J key, and 0 points for pressing the H key). However, directly looking at the scores is not intuitive enough. Softmax then does two things: First, it amplifies the differences, converting the scores into a probability-like form, making actions with higher scores more prominent. Second, it makes the sum of the probabilities equal to 1, ensuring that the sum of the probabilities of all actions is exactly 100%. For example, after Softmax processing, the 3-point M key press becomes 70%, and the probabilities of the other three cases combined are only 30%. In other words, the model believes that the finger joint has a 70% probability of pressing the M key next, while the probabilities of other actions are very low. Finally, based on the softmax output, the predicted probability of each keyboard key being pressed is obtained. The character corresponding to the key position with the highest probability is the predicted character the user will input. In the scenario shown in Figure 7, the most probable keystroke positions can be predicted in advance as m and n (approximately 0.4 seconds). Therefore, the simulated keyboard positions m and n are enlarged and highlighted, indicating that m and n have been pre-selected. This significantly improves typing speed and fluency.

[0095] [Methods for determining valid actions]

[0096] As described above, it is necessary to determine the valid motion of the user's hand during the process of recognizing hand joint points. The method for determining a valid motion will be described below by taking a typing motion as an example. First, the relative positions of each hand joint point or data representing such a relative positional relationship (for example, the curvature range of the connecting line of hand joint points, etc.) is compared with template data stored in the intelligent information processing apparatus 120, to obtain a comparison result value Result1 (0 < Result1 ≤ 1). In addition, a lightweight large language model running on the intelligent information processing apparatus 120 is also used to obtain a predicted motion result value Result2 (0 < Result2 ≤ 1). Then, by setting appropriate coefficients β1 and β2, the comparison result value Result1 and the predicted motion result value Result2 are combined to obtain a motion prediction value Result = β1*Result1+β2*Result2. When the motion prediction value Result ≥ 1, it indicates that the motion conforms to the pre-judged template motion (typing in this case). In this case, the portable intelligent office auxiliary device 100 in the standby state will switch to the state of simulating keyboard input.

[0097] [Embodiments of other office auxiliary functions]

[0098] As mentioned above, the portable intelligent office auxiliary device 100 according to the embodiments of the present disclosure may further include an office auxiliary module 126 with customized office auxiliary functions as required by the user. For example, the intelligent information processing apparatus 120 of the portable intelligent office auxiliary device 100 may be provided with a workflow assistance processing module, so as to intelligently assist the user in work schedule setting and trigger of automatic workflows of personal Agents in accessed PCs, servers and / or personal portable devices. In this case, after turning on the portable intelligent office auxiliary device 100, the user manually sets or updates the daily work schedule. The input text content can automatically trigger part of preset AI Agent workflows (such as automatic prediction and statistics of current work completion, short reminder of deadlines, and brief automatic reply in combination with applications such as email, Teams and WeChat). If the schedule of a certain day includes a meeting option, the keyword "meeting" will cause the AI Agent to trigger the attention mode of the accessed device. When there is an emergency demand from other office work channels, such as emails, Teams and other applications, the user will be reminded by judging the sender and the importance and urgency of the sent content (such as response to sudden urgent and important tasks), and automatically reply to the sender briefly. Then, the workflow assistance processing module can also update the work schedule according to the automatically determined priority.

[0099] In addition, the portable intelligent office auxiliary device according to the present invention can be configured as follows.

[0100] (1) A portable intelligent office assistance device, characterized in that the intelligent office assistance device comprises:

[0101] The scanning device is equipped with an image sensor, which is capable of scanning the surrounding environment and extracting facial features of people and object features of objects in the surrounding environment.

[0102] An intelligent information processing device receives facial feature information and item feature information from the scanning device, and includes a facial recognition module, a digital key module, and an item management module.

[0103] The facial recognition module performs facial recognition based on the facial feature information and determines authorized users existing in the surrounding environment based on the facial recognition result; the digital key module obtains authorization for the authorized user to access another office device using the portable smart office auxiliary device based on the facial recognition result; the item management module summarizes and processes the item feature information and infers the item search result based on the item search prompts provided by the authorized user and the summarized item feature information; and

[0104] A display device is provided that can display the output of the intelligent information processing device.

[0105] (2) The portable intelligent office auxiliary device described in (1) above is characterized in that the image sensor includes an image acquisition unit and a feature extraction unit, the image acquisition unit acquires image information of the surrounding environment, and the feature extraction unit extracts facial feature information and object feature information based on the image information.

[0106] (3) The portable intelligent office auxiliary device described in (1) above is characterized in that the digital key module accesses the other office device by using WebAssembly technology based on the result of the face recognition to obtain access authorization for hardware and software facilities at a specified level.

[0107] (4) The portable intelligent office auxiliary device described in (1) above, characterized in that the information summarization and processing includes at least:

[0108] Identifying and processing key items among items; and

[0109] Information compression storage processing is performed on the compression and storage of the feature information of objects in the surrounding environment based on spatial dimensions.

[0110] (5) The portable intelligent office auxiliary device described in (4) above is characterized in that, in the key item determination process, the item management module determines the surrounding auxiliary items existing around the authorized user based on the face recognition result of the face recognition module, and determines the key items among the surrounding auxiliary items based on the item feature information of the surrounding auxiliary items.

[0111] (6) The portable intelligent office auxiliary device described in (5) above is characterized in that the item management module identifies the surrounding accessories of the authorized user based on the contact frequency between the authorized user and the items existing in the surrounding environment.

[0112] (7) The portable intelligent office auxiliary device described in (5) above is characterized in that the item management module of the intelligent information processing device determines the key item based on the key item feature value calculated based on the item feature information of the surrounding auxiliary items.

[0113] (8) The portable intelligent office auxiliary device described in (4) above is characterized in that the item management module uses a lightweight large language model running on the intelligent information processing device to perform the information compression and storage processing according to the item-by-item dependency relationship between the key item and other items.

[0114] (9) The portable intelligent office auxiliary device described in (8) above is characterized in that, in the information compression and storage processing, the item management module extracts the item layer-by-layer dependency relationship based on the spatial dimension correlation between the key item and other items, and constructs a map linked list for compressing and storing the item feature information based on the item layer-by-layer dependency relationship.

[0115] (10) The portable intelligent office auxiliary device described in (9) above is characterized in that, in the information compression and storage process, the item management module filters a portion of the item feature information according to feature saliency to determine the compressed feature information in the item feature information of the other items, and stores the compressed feature information in the map linked list.

[0116] (11) The portable intelligent office auxiliary device described in (9) above is characterized in that, in the map chain, the item feature information associated with an item corresponds to a storage node, and the reference time is used as the head of the map chain.

[0117] (12) The portable intelligent office auxiliary device described in (4) above is characterized in that the information summarization processing further includes information update processing for updating the compressed and stored item feature information, and

[0118] When the surrounding accessories that exist around the authorized user change or disappear, or when new surrounding accessories appear around the authorized user, the item management module performs the information update process.

[0119] (13) The portable intelligent office auxiliary device described in (11) above is characterized in that, when the characteristics of the peripheral accessories other than the key item change or disappear, or when a new peripheral accessory appears, the item management module performs the information update process to update the compressed feature information stored in the map chain, and records the time of these changes relative to the reference time; and

[0120] When the characteristics of the key item in the scene change or a new surrounding accessory is identified as the new key item, the item management module performs the information update process to update the compressed feature information stored in the map linked list, and stores the change time in the head of the map linked list as the new base time.

[0121] (14) The portable intelligent office auxiliary device described in (11) above is characterized in that the item management module adds the item search prompt information to the input of the micro-generation model running on the intelligent information processing device, expands the compressed feature information stored in the map chain, and thus obtains the item search result.

[0122] (15) The portable intelligent office auxiliary device described in (14) above is characterized in that the intelligent office auxiliary device further includes an input device capable of inputting the item search prompt information, and the display device displays the item search result to the authorized user, the authorized user inputs correction prompt information through the input device, and the item management module obtains the corrected item search result based on the current item search result and the correction prompt information.

[0123] (16) The portable intelligent office auxiliary device described in any one of (1) to (15) above is characterized in that the image sensor is an intelligent image sensor that extracts only the facial feature information and the object feature information in the surrounding environment.

[0124] (17) The portable intelligent office auxiliary device according to any one of (1) to (14) above, characterized in that the intelligent information processing device further includes a joint point motion recognition module and an input operation capture module, wherein,

[0125] The image sensor can also extract joint information of people in the surrounding environment. The joint motion recognition module performs motion recognition on the joint motion of the authorized user based on the joint information from the image sensor and the face recognition result from the face recognition module. When the authorized user's valid motion is recognized, the intelligent office auxiliary device enters the input operation capture mode. In the input operation capture mode, the input operation capture module of the intelligent office auxiliary device generates simulated motion information that matches the valid motion.

[0126] (18) The portable intelligent office auxiliary device according to (17) above is characterized in that, in the input operation capture mode, the intelligent information processing device controls the display device to display a simulated keyboard and / or simulated touchpad for input to the other office device, and obtains the simulated action information of the simulated action performed on the simulated keyboard and / or simulated touchpad based on the key point information by using the action prediction model running on the intelligent information processing device, and uses the simulated action information to control the input to the other office device and control the display device to display the simulated action.

[0127] (19) The portable intelligent office auxiliary device described in (18) above is characterized in that it further includes a voice input device, through which the authorized user inputs voice control information, and the intelligent information processing device parses the voice control information and combines the parsed voice control information with the simulated action information to control the input to the other office device and control the display device to display the simulated action.

[0128] (20) The portable intelligent office auxiliary device described in (17) above is characterized in that the joint information is the joint information of the finger joints of a person in the surrounding environment.

[0129] (21) The portable intelligent office auxiliary device described in any one of (1) to (15) above is characterized in that the item feature information includes at least one of the item's location, color distribution frequency, texture roughness, area, shape and angular features.

[0130] (22) The portable intelligent office auxiliary device described in (19) above is characterized in that it further includes an input device capable of inputting prompt information, and the intelligent information processing device further includes an office auxiliary module, which, based on the recognition result of the action recognition by the joint action recognition module, grants access rights to a specified Internet address to the portable intelligent office auxiliary device, and

[0131] Based on the prompt information input by the authorized user using the input device and / or the voice control information input using the voice input device, the office assistance module automatically generates office support documents using a lightweight large language model and a micro-generation model running on the intelligent information processing device.

[0132] (23) The portable intelligent office auxiliary device described in (22) above, wherein the designated Internet address is a specific corporate RAG knowledge base and / or a designated Internet site.

[0133] Those skilled in the art will understand that although this disclosure has been described above with reference to embodiments, it is not limited to the above-described embodiments, and various modifications, combinations, sub-combinations, and alterations can be made according to design requirements and other factors, as long as they fall within the scope of the appended claims or their equivalents. Furthermore, the effects described in this specification are merely illustrative, and the effects of this disclosure are not limited to those described herein.

[0134] Citation of relevant applications

[0135] This application claims the benefit of Chinese Patent Application No. 202510352135.2, filed on March 24, 2025 with the State Intellectual Property Office of the People's Republic of China, the entire contents of which are hereby incorporated by reference.

Claims

1. A portable intelligent office auxiliary device, characterized in that, The portable intelligent office auxiliary device includes: The scanning device is equipped with an image sensor, which is capable of scanning the surrounding environment and extracting facial features of people and object features of objects in the surrounding environment. An intelligent information processing device receives facial feature information and item feature information from the scanning device, and includes a facial recognition module, a digital key module, and an item management module. The facial recognition module performs facial recognition based on the facial feature information and determines authorized users in the surrounding environment based on the facial recognition result; the digital key module obtains authorization for the authorized user to access another office device using the portable smart office auxiliary device based on the facial recognition result; the item management module summarizes and processes the item feature information and infers the item search result based on the item search prompts provided by the authorized user and the summarized item feature information; and A display device is provided that can display the output of the intelligent information processing device.

2. The portable intelligent office auxiliary device according to claim 1, characterized in that, The image sensor includes an image acquisition unit and a feature extraction unit. The image acquisition unit acquires image information of the surrounding environment, and the feature extraction unit extracts facial feature information and object feature information based on the image information.

3. The portable intelligent office auxiliary device according to claim 1, characterized in that, The digital key module uses WebAssembly technology to access another office device based on the result of facial recognition to obtain access authorization for hardware and software facilities at a specified level.

4. The portable intelligent office auxiliary device according to claim 1, characterized in that, The information summarization and processing includes at least the following: Identifying and processing key items among items; and Information compression storage processing is performed on the compression and storage of the feature information of objects in the surrounding environment based on spatial dimensions.

5. The portable intelligent office auxiliary device according to claim 4, characterized in that, In the critical item identification process, the item management module determines the surrounding auxiliary items existing around the authorized user based on the face recognition result of the face recognition module, and determines the critical items among the surrounding auxiliary items based on the item feature information of the surrounding auxiliary items.

6. The portable intelligent office auxiliary device according to claim 5, characterized in that, The item management module identifies the authorized user's surrounding items based on the frequency of contact between the authorized user and items in the surrounding environment.

7. The portable intelligent office auxiliary device according to claim 5, characterized in that, The item management module of the intelligent information processing device determines the key item based on the key item feature value calculated from the item feature information of the surrounding attached items.

8. The portable intelligent office auxiliary device according to claim 4, characterized in that, The item management module uses a lightweight large language model running on the intelligent information processing device to compress and store the information according to the hierarchical dependency relationship between the key item and other items.

9. The portable intelligent office auxiliary device according to claim 8, characterized in that, In the information compression and storage process, the item management module extracts the layer-by-layer dependency relationship of the key item and other items based on the spatial dimension correlation, and constructs a map linked list for compressing and storing the item feature information based on the layer-by-layer dependency relationship.

10. The portable intelligent office auxiliary device according to claim 9, characterized in that, In the information compression and storage process, the item management module filters a portion of the item feature information based on feature saliency to determine the compressed feature information in the item feature information of other items, and stores the compressed feature information in the map linked list.

11. The portable intelligent office auxiliary device according to claim 9, characterized in that, In the map linked list, the item feature information associated with an item corresponds to a storage node, and the reference time is used as the head of the map linked list.

12. The portable intelligent office auxiliary device according to claim 4, characterized in that, The information summarization process also includes information update processing to update the compressed and stored item feature information, and When the surrounding accessories that exist around the authorized user change or disappear, or when new surrounding accessories appear around the authorized user, the item management module performs the information update process.

13. The portable intelligent office auxiliary device according to claim 11, characterized in that, When the characteristics of items other than the key item among the surrounding auxiliary items change or disappear, or when new surrounding auxiliary items appear, the item management module performs the information update process to update the compressed feature information stored in the map chain, and records the time of these changes relative to the base time. and When the characteristics of the key item in the scene change or a new surrounding accessory is identified as the new key item, the item management module performs the information update process to update the compressed feature information stored in the map linked list, and stores the change time in the head of the map linked list as the new base time.

14. The portable intelligent office auxiliary device according to claim 11, characterized in that, The item management module adds the item search prompt information to the input of the micro-generative model running on the intelligent information processing device, and expands the compressed feature information stored in the map chain to obtain the item search result.

15. The portable intelligent office auxiliary device according to claim 14, characterized in that, The portable intelligent office auxiliary device also includes an input device that can input the item search prompt information, and the display device displays the item search results to the authorized user. The authorized user inputs correction prompt information through the input device, and the item management module obtains the corrected item search results based on the current item search results and the correction prompt information.

16. The portable intelligent office auxiliary device according to any one of claims 1 to 15, characterized in that, The image sensor is an intelligent image sensor that extracts only the facial feature information and the object feature information in the surrounding environment.

17. The portable intelligent office auxiliary device according to any one of claims 1 to 14, characterized in that, The intelligent information processing device further includes a joint point motion recognition module and an input operation capture module, wherein... The image sensor can also extract joint information of people in the surrounding environment. The joint motion recognition module performs motion recognition on the joint motion of the authorized user based on the joint information from the image sensor and the face recognition result from the face recognition module. When the authorized user's valid motion is recognized, the portable smart office auxiliary device enters the input operation capture mode. In the input operation capture mode, the input operation capture module of the portable smart office auxiliary device generates simulated motion information that matches the valid motion.

18. The portable intelligent office auxiliary device according to claim 17, characterized in that, In the input operation capture mode, the intelligent information processing device controls the display device to display a simulated keyboard and / or simulated touchpad for input to the other office device, and obtains simulated action information of the simulated action performed on the simulated keyboard and / or simulated touchpad based on the key point information by using a motion prediction model running on the intelligent information processing device, and uses the simulated action information to control input to the other office device and control the display device to display the simulated action.

19. The portable intelligent office auxiliary device according to claim 18, characterized in that, It also includes a voice input device, through which the authorized user inputs voice control information. The intelligent information processing device parses the voice control information and combines the parsed voice control information with the simulated action information to control the input to the other office equipment and control the display device to display the simulated action.

20. The portable intelligent office auxiliary device according to claim 17, characterized in that, The joint information refers to the joint information of the finger joints of a person in the surrounding environment.

21. The portable intelligent office auxiliary device according to any one of claims 1 to 15, characterized in that, The item feature information includes at least one of the following: item location, color distribution frequency, texture roughness, area, shape, and angular features.

22. The portable intelligent office auxiliary device according to claim 19, characterized in that, It also includes an input device capable of inputting prompts, and the intelligent information processing device further includes an office assistance module, which, based on the action recognition result of the joint action recognition module, grants access to a specified internet address to the portable intelligent office assistance device. Based on the prompt information input by the authorized user using the input device and / or the voice control information input using the voice input device, the office assistance module automatically generates office support documents using a lightweight large language model and a micro-generation model running on the intelligent information processing device.

23. The portable intelligent office auxiliary device according to claim 22, wherein, The specified internet address is a specific corporate RAG knowledge base and / or a specified internet site.

Images