Optical sensing device
By using an image extraction device and a flexible light shield in an optical sensing device, the problem that the annular cushion is difficult to fit uneven skin is solved, the sensing accuracy and reference value are improved, and the morphological changes of arteriovenous fistula are adapted to achieve Efficient optical sensing.
Patent Information
- Application Number
- CN202010249088.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-03-05
- Filing Date
- 2020-04-01
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2040-04-01
AI Technical Summary
When the existing optical sensing device senses arteriovenous fistula, the annular pads are difficult to fit tightly into the uneven skin surface, resulting in limited light shading effect, affecting the sensing accuracy, and the deformation and curvature of the arteriovenous fistula are increased. This makes long-term sensing difficult and the sensing value low reference value.
The image extraction device and a flexible light hood are used. The image extraction device extracts the image of the optical sensor sensing target to increase the correlation of sensing data. The flexible light hood is closely attached along the outline of the sensing target to provide effective light shading and shield external light. interference.
The sensing accuracy and reference value of the optical sensing device are improved, the effectiveness of long-term sensing is ensured, and the sensing needs of different locations and skin morphology are adapted.
Smart Images

Figure CN113349755B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an optical sensing device, and in particular to an optical sensing device suitable for sensing physiological parameters. Background Art
[0002] At present, there are applications that use optical sensing to sense physiological parameters, such as using photoplethysmography (PPG) technology to sense pulse, blood flow, etc. Its optical sensor usually emits light by itself and senses this light. Therefore, when there is interference from external light, the accuracy of sensing will be affected. At present, there is an optical sensing device for sensing the state of arteriovenous fistula on the market, and the device body is provided with an exposed optical sensor and an annular cushion surrounding the optical sensor. When in use, the optical sensor is directed toward the arteriovenous fistula of the patient and the device body is moved so that the optical sensor can be as close to the arteriovenous fistula as possible and the annular cushion is as close to the skin as possible to prevent ambient light from entering the inner side of the annular cushion and interfering with the sensing of the optical sensor. However, the annular cushion is usually in the form of a sheet (or a thin plate), and its elastic deformation is limited, and the skin surface contour is usually uneven, so that the annular cushion is not easy to completely fit the skin. Furthermore, the actual arteriovenous fistula is usually not flat, which makes the skin surface contour at that location even more uneven, resulting in the very limited light shielding effect of the annular cushion, causing the optical sensor to receive ambient light during sensing, making the sensing invalid or the reference value of the sensing value low. Furthermore, the actual arteriovenous fistula usually deforms, increases in curvature, etc. over time, and the light shielding function of the annular cushion may completely fail, making the sensing of the optical sensing device difficult to implement, or the sensing fails and the sensing value is meaningless. Furthermore, the aforementioned deformation, increased curvature, etc. phenomena also make it difficult for the optical sensing device to sense at the same position for a long time to observe the changes in the arteriovenous fistula. In addition, the sensing values obtained by sensing at different positions of the arteriovenous fistula and sensing by the optical sensing device at different horizontal states are all different. The current optical sensing device only displays or records the sensing value, and cannot distinguish the sensing value according to the sensing posture (such as the aforementioned sensing position, the horizontal state of the device during sensing, etc.), which makes the simple sensing value have a low reference value in the future. Summary of the invention
[0003] In view of the problems in the prior art, an object of the present invention is to provide an optical sensing device, which can utilize an image capturing device to capture an image of a target sensed by the optical sensing device, so as to increase the correlation between sensing data and the sensed target.
[0004] The optical sensing device according to the present invention comprises a device housing, an optical sensor and an image capturing device. The optical sensor is disposed on the device housing and has a sensing direction, and the optical sensor senses in the sensing direction. The image capturing device is disposed on the device housing and has an capturing direction, and the sensing direction is in the same direction as the capturing direction. Thus, the optical sensing device can utilize the image capturing device to capture the image of the target sensed by the optical sensor, so as to increase the correlation between the sensing data and the sensing target, and thus increase the reference value of the sensing data in the future, thereby solving the problem in the prior art that the optical sensing device only records the sensing value, so that the simple sensing value has a low reference value in the future.
[0005] Another object of the present invention is to provide an optical sensing device, wherein the optical sensor is surrounded by a flexible light shielding cover, which can increase the closeness with the sensing target and effectively shield the influence of external light on the sensing of the optical sensor.
[0006] The optical sensing device according to the present invention comprises a device housing, an optical sensor and a flexible light shield. The optical sensor is disposed on the device housing. The flexible light shield surrounds the optical sensor and is disposed on the device housing. Thus, the flexible light shield can closely fit the sensing target along the contour of the sensing target to provide the optical sensor with sufficient light shielding effect, thereby solving the problem in the prior art that the annular soft pad is difficult to closely fit the sensing target with an uneven surface, and also overcoming the limitation that the annular soft pad in the prior art can only be applied to sensing targets with flat surfaces.
[0007] The advantages and spirit of the present invention can be further understood through the following detailed description of the invention and the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is a schematic diagram of an optical sensing device according to an embodiment;
[0009] Figure 2 for Figure 1 A functional block diagram of an optical sensing device;
[0010] Figure 3 for Figure 1 A cross-sectional view of the optical sensing device along line XX;
[0011] Figure 4 for Figure 3 A cross-sectional view of an optical sensing device applied to sense an arteriovenous fistula beneath the skin;
[0012] Figure 5 A schematic diagram showing that the indicator mark projected by the light source of the optical sensing device is a cross;
[0013] Figure 6 A schematic diagram showing a circle for the indication;
[0014] Figure 7 A schematic diagram with arrows marking the indication;
[0015] Figure 8 A schematic diagram is marked with a polygon for this indication;
[0016] Fig. 9 is a cross-sectional view of an optical sensing device according to another embodiment;
[0017] Fig.10 A schematic diagram of a controller recording content format of an optical sensing device and an example thereof.
[0018] Explanation of symbols
[0019] 1: Optical sensing device
[0020] 12: Device housing
[0021] 122: Boss
[0022] 14: Optical sensor
[0023] 14a: Sensing direction
[0024] 16: Image capture device
[0025] 16a,16b: Extraction direction
[0026] 18,19: Flexible sunshade
[0027] 18a, 18b: end
[0028] 192: Pipe sleeve
[0029] 194: Annular lip
[0030] 20: Controller
[0031] 20a: Memory
[0032] 22: Electronic level
[0033] 24: Display screen
[0034] 26: Light Source DETAILED DESCRIPTION
[0035] See also Figures 1 to 3 An optical sensing device 1 according to an embodiment includes a device housing 12, an optical sensor 14, an image capturing device 16, a flexible light shield 18 and a controller 20; Figure 3In order to simplify the drawings, the optical sensor 14, the image capturing device 16 and the controller 20 are all represented by solid blocks. The optical sensor 14 is disposed on the device housing 12 and has a sensing direction 14a (indicated by an arrow in the figure), and the optical sensor 14 performs sensing in the sensing direction 14a. The image capturing device 16 is disposed on the device housing 12 and has an capturing direction 16a (indicated by an arrow in the figure). The image capturing device 16 performs image capturing in the capturing direction 16a. The flexible light shield 18 surrounds the optical sensor 14 and is disposed on the device housing 12. The controller 20 is disposed in the device housing 12 and is electrically connected to the optical sensor 14 and the image capturing device 16 (indicated by an arrow in the figure). Figure 3 , indicated by dotted lines). The controller 20 has a recording function, and in practice it can be implemented by a circuit board module (e.g., including a circuit board and a processor, a memory 20a and other required electronic components disposed thereon). The controller 20 can receive an optical signal related to a blood flow via the optical sensor 14, and can receive and record an image in the memory 20a via the image capture device 16. The flexible light shield 18 has sufficient flexibility to fit the sensing target closely along the contour of the sensing target, so as to provide the optical sensor 14 with sufficient light shielding effect. The sensing direction 14a and the capturing direction 16a are in the same direction (or the optical sensor 14 and the image capture device 16 are in the same direction), so the image capture device 16 can capture an image including at least a portion of the sensing target to provide image information about the sensing target. On the other hand, the image capture device 16 extracting an image about the sensing target helps to increase the correlation between the sensing data and the sensing target, that is, to increase the reference value of the sensing data in the future.
[0036] In addition, in this embodiment, Figure 3 As shown, the optical sensor 14 and the image capturing device 16 are both arranged downward, and the sensing direction 14a is parallel to the capturing direction 16a, but the implementation is not limited to this. For example, in one embodiment, the capturing direction 16b of the image capturing device 16 (indicated by a dotted arrow in FIG. 1 ) is Figure 3 The image capturing device 16 still faces downward, but is biased toward the front of the optical sensor 14; wherein, although the capturing direction 16b forms an acute angle with the sensing direction 14a, the capturing direction 16b and the sensing direction 14a are still in the same direction. At this time, the image captured by the image capturing device 16 can include images of more parts of the sensing target, thereby providing more information about the sensing target, that is, improving the reusability or reference value of the data.
[0037] In the present embodiment, the optical sensor 14 may include at least one light source (e.g., a green light source, a red light source, an infrared light source) and a light detector disposed adjacent thereto. The light source emits light, and the light is received by the light detector after being reflected by the sensing target. The controller 20 obtains sensing results about the sensing target by analyzing the relationship between the light emitted by the light source and the light received by the light detector, for example but not limited to based on optical plethysmography technology. In practice, the optical sensor 14 may adopt a general optical sensor based on PPG technology, so its structural details are not described separately. The image capture device 16 may include, for example but not limited to, a photosensitive coupling element or a complementary metal oxide semiconductor photosensitive element. The flexible light shield 18 is a hollow thin shell structure and may be made of rubber, so it may easily provide sufficient flexibility to fit closely to the sensing target along the contour of the sensing target; on the other hand, if Figure 3 As shown, this hollow shell structure can be regarded as a double-layer structure in structure, and in principle is only connected at the upper and lower ends 18a, 18b (i.e., the side wall edges of the double-layer structure). There is a gap between the side walls of the double-layer structure, so that the hollow shell structure as a whole is also flexible. The memory 20a can store the aforementioned extracted images, sensing data (including date, time, light signal or information obtained by analyzing the light signal). In addition, the optical sensing device 1 can further include other components (such as a power module, a communication module, a connection interface (such as but not limited to Bluetooth, WiFi, USB), etc.).
[0038] Please also refer to Figure 4 , which is a cross-sectional view of the optical sensing device 1 used to sense an arteriovenous fistula (located at a skin bulge) below the skin; the arteriovenous fistula itself is not shown in the figure. The flexible light shield 18 obviously protrudes relative to the optical sensor 14 in the sensing direction 14a, so before the optical sensor 14 contacts the skin, the side wall of the flexible light shield 18 can bend or bend in accordance with the surface morphology of the skin, so that the flexible light shield 18 can effectively fit the skin tightly, so that the optical sensor 14 will not be disturbed by external light (such as ambient light) when sensing the arteriovenous fistula. In addition, before, during, and after the optical sensor 14 senses the arteriovenous fistula, the image capture device 16 can be used to capture and record images of the skin surface morphology above or near the arteriovenous fistula in the memory 20a, so as to associate the sensing data with the sensing target for future reference.
[0039] In addition, in this embodiment, the optical sensing device 1 further includes an electronic level 22, which is disposed on the device housing 12 and electrically connected to the controller 20. The controller 20 receives and records the level data (e.g., the tilt angle) from the electronic level 22 in the memory 20a. For example, after the optical sensing device 1 is placed on the sensing target, the controller 20 records the level data at this time in the memory 20a, thereby being able to generate a correlation between the sensing data and the level data for future reference. In practice, the electronic level 22 can be implemented by a digital gyroscope chip with more than six axes, which can also be integrated into the circuit board module of the controller 20.
[0040] In addition, in this embodiment, the optical sensing device 1 further includes a display screen 24 (such as but not limited to a liquid crystal display module), which is exposed from the device housing 12 and electrically connected to the controller 20. The controller 20 controls the display screen 24 to display a level information (which can be represented by one or more icons) about the level data, the sensing state, the sensing result, the time, the date, etc., or even to display the image captured by the image capturing device 16 in real time. Moreover, in practice, the display screen 24 can be implemented as a touch display panel, so that the controller 20 can also receive user input through the touch display panel. Moreover, in practice, the optical sensing device 1 can further include a plurality of buttons, which are exposed from the device housing 12 (such as arranged adjacent to the display screen 24), for user input (such as input data, control instructions, etc.).
[0041] In addition, in this embodiment, the optical sensing device 1 further includes a light source 26 (for example but not limited to), which is disposed on the device housing 12 and is electrically connected to the controller 20. The controller 20 controls the light source 26 to project an indication mark in the sensing direction 14a. The indication mark may be a cross, a circle, an arrow or a polygon, such as Figures 5 to 8 As shown; in practice, the indicator mark can also be other graphics (including but not limited to text, numbers, symbols, etc.), or a combination of the above marks, such as a cross plus a circle. In practice, the light source 26 can be realized by a light source (such as but not limited to a light emitting diode) in combination with a light guide column, an optical fiber or a lens (for forming the indicator mark, such as but not limited to coating the end face of the light guide column or the optical fiber, or the surface of the lens with a light-shielding ink to form a pattern that conforms to the indicator mark). When the indicator mark is projected onto or near the sensing target (such as the skin above the arteriovenous fistula), it can provide a reference for the user to move the optical sensing device 1, thereby more accurately moving the optical sensing device 1 toward the sensing target. In this embodiment, the light source 26 is disposed adjacent to the optical sensor 14, so that the indicator mark can be closer to the sensing target, which is convenient for the user to accurately move the optical sensing device 1. In practice, the projection direction of the light source 26 can be slightly tilted toward the sensing direction 14a, so that the projected indicator mark can be closer to the sensing target.
[0042] like Figure 1 , Figure 3 and Figure 4 As shown, in this embodiment, the device housing 12 further has a boss 122, which protrudes relative to the body of the device housing 12 in the sensing direction 14a. The optical sensor 14 is located on the boss 122, which helps to reduce the interference between the optical sensing device 1 and the sensing target (and the structure nearby), so that the optical sensor 14 can be close to or closely fit the sensing target. In this embodiment, the light source 26 is also located on the boss 122, which can make it easier to project the indicator mark near the sensing target. In addition, the image capture device 16 is arranged on the device housing 12 adjacent to the boss 122, and the flexible light shield 18 also surrounds the image capture device 16. This structural configuration can prevent the flexible light shield 18 from shielding the image capture device 16 from capturing the image of the sensing target, and can also be used to capture the image through the image capture device 16 after the flexible light shield 18 is closely fitted to the skin, confirming or recording the actual relative position of the optical sensing device 1 (or the boss 122) and the sensing target at this time (for example, by temporarily providing the light required for capturing the image through other light sources). In addition, the flexible light shield 18 obviously protrudes relative to the boss 122 in the sensing direction 14 a and also protrudes relative to the optical sensor 14 , so before the optical sensor 14 contacts the skin, the flexible light shield 18 can effectively and closely fit the skin.
[0043] In addition, in the present embodiment, the flexible light shield 18 as a whole has a cross-sectional profile in the sensing direction 14a (or a cross-section on a reference plane perpendicular to the sensing direction 14a), and the cross-sectional profile is rectangular in accordance with the profile of the boss 122, but in practice, the cross-sectional profile may also be circular, elliptical, or other polygonal. Furthermore, in practice, the flexible light shield 18 may be designed to be replaceable (for example, removably mounted on the device housing 12) to facilitate the selection of flexible light shields with different structural profiles to accommodate different sensing targets (for example, different skin morphologies above the arteriovenous fistula). Furthermore, in practice, the flexible light shield 18 may also be a flexible light shield with a non-hollow thin shell structure. Fig. 9As shown, another flexible light shield 19 replaces the flexible light shield 18 and is disposed on the device housing 12. The flexible light shield 19 includes a tube sleeve portion 192 and an annular lip portion 194. The tube sleeve portion 192 is disposed on the device housing 12, and the annular lip portion 194 extends outward from the tube sleeve portion 192. The flexible light shield 19 can be implemented by a thin shell structure made of rubber, and it can also easily provide sufficient flexibility to closely fit the sensing target along the contour of the sensing target. Among them, the annular lip portion 194 extending outward can effectively closely fit the sensing target, so that the flexible light shield 19 can also provide the same effective light shielding effect for the optical sensor 14 as the aforementioned flexible light shield 18. For other descriptions of the flexible light shield 19, please refer to the relevant descriptions of the flexible light shield 18 and its variations (including the contour of the flexible light shield 18, the relative arrangement relationship between the flexible light shield 18 and other components, etc.), which will not be repeated.
[0044] The use of the optical sensing device 1 can be implemented by the following steps, but the actual application is not limited to this. First, the subject's arm is placed flat and the optical sensing device 1 is turned on. The optical sensor 14, the image capture device 16, the electronic level 22 and the light source 26 are turned on. The indicator mark generated by the light source 26 is projected onto the measured point (or small measured area, i.e., the sensing target) on the arm or its vicinity. In practice, for example, when the optical sensing device 1 is used to sense the blood flow in the arteriovenous fistula for dialysis, the selection of the measured point should be set to be away from the upper needle and lower needle to avoid the influence of the turbulence that is easily generated there on the sensing result.
[0045] Next, the optical sensing device 1 is moved so that the optical sensor 14 can be aligned or moved to the test point, and the flexible light shield 18 can be closely attached to the skin near the test point to prevent external light from entering the flexible light shield 18 and affecting the sensing result of the optical sensor 14. In practice, patients often cover the skin above the arteriovenous fistula with plastic wrap to protect the anesthetic application area, which can still effectively and closely attach the flexible light shield 18. In addition, at this time, the display screen 24 on the optical sensing device 1 can display the current attachment angle state (for example, displaying the current level data generated by the electronic level 22).
[0046] The optical sensor 14 continuously senses the blood flow in the arteriovenous fistula. After the controller 20 determines that the optical signal of the blood flow in the arteriovenous fistula is stable (for example, it remains stable for a short period of time), the controller 20 can calculate the blood flow rate based on the optical signal and record the blood flow rate in the memory 20a. The controller 20 also simultaneously records the horizontal data at this time, the image captured by the image capture device 16 (which can be captured before the flexible light shield 18 is tightly attached to the skin, or when the indicator mark is projected onto the test point), time, date, etc. (in the memory 20a). The operation of the aforementioned controller 20 can be automatically executed by the firmware written into the controller 20. In principle, the user only needs to turn on the optical sensing device 1, move the optical sensing device 1 to align with the test point, or after aligning with the test point, trigger the optical sensor 14 to start sensing. The overall operation of the optical sensing device 1 is simple and does not require professional knowledge. It is very suitable for patients to detect themselves, saving patients the time to go to specialized medical institutions. In practice, if Fig.10 As shown, each data recorded by the controller 20 in the memory 20a may include an identification code (for example, an identification code corresponding to different patient settings), a sensing time (for example, after the light signal becomes stable, the system time is read from the optical sensing device 1), blood oxygen saturation (SPO2) (for example, a value can be input by the user), systolic pressure (for example, a value can be input by the user), diastolic pressure (for example, a value can be input by the user), blood flow (the controller 20 calculates the value according to the light signal), signal extraction time (for example, the controller 20 calculates according to the system time), an image (i.e., the extracted image, recorded in the file name The image can show the surface state of the measured point, such as whether there are other foreign objects on the skin, the skin condition itself, such as skin color, hair, etc.), horizontal state (for example, according to horizontal data), instrument number (for example, the serial number preset by the optical sensing device 1), firmware version (that is, the firmware version running on the optical sensing device 1), remarks (for users to add, such as noting that the patient's sensing area skin is covered with plastic wrap), etc. This can increase the correlation between blood flow and other data items (such as blood oxygen saturation, systolic pressure, diastolic pressure, etc.) (for example, the relationship between blood pressure and blood flow), and also help to improve the reusability or reference value of data. The above-mentioned input value can be input by the user, for example, but not limited to, through a display screen with a touch function, or through the buttons provided by the optical sensing device 1 and the display screen 24. In addition, in practice, the optical sensing device 1 can transmit the records stored in the memory 20a of the controller 20 to an external electronic device (such as a computer host) through a communication module, which can be used for data storage, data collection, further analysis or other data processing.
[0047] The above descriptions are only preferred embodiments of the present invention. All equivalent changes and modifications made according to the claims of the present invention should fall within the scope of the present invention.
Claims
1. An optical sensing device, characterized in that: Include: Device housing; An optical sensor is disposed on the device housing and has a sensing direction, and the optical sensor performs sensing in the sensing direction; An image capturing device is disposed on the device housing and has a capturing direction, the sensing direction being in the same direction as the capturing direction; as well as A flexible light shield is disposed on the device housing and surrounds the optical sensor and the image capturing device. Wherein, the flexible sunshade is a hollow thin shell structure; or The flexible light shield includes a tube sleeve portion and an annular lip portion, wherein the tube sleeve portion is arranged on the device housing, and the annular lip portion extends outward from the tube sleeve portion. The flexible light shield has a cross-sectional profile in the sensing direction, and the cross-sectional profile is circular, elliptical or polygonal.
2. The optical sensing device as claimed in claim 1, further comprising a light source, which is disposed on the device housing and projects an indication mark in the sensing direction, wherein the indication mark is in the shape of a cross, a circle, an arrow or a polygon.
3. The optical sensing device as described in claim 1 further includes a light source, which is disposed on the device housing and projects an indication mark in the sensing direction, wherein the device housing has a boss, the optical sensor and the light source are located on the boss, the light source is adjacent to the optical sensor, and the image capture device is adjacent to the boss.
4. The optical sensing device as described in claim 1 further includes a controller, which is disposed in the device housing and is electrically connected to the optical sensor and the image capture device, wherein the controller has a memory, and the controller receives optical signals related to blood flow through the optical sensor and receives and records images in the memory through the image capture device.
5. The optical sensing device as described in claim 4 further comprises an electronic level disposed on the device housing, wherein the controller is electrically connected to the electronic level, and the controller receives and records the level data from the electronic level in the memory. 6 . The optical sensing device as claimed in claim 5 , further comprising a display screen exposed from the device housing, wherein the controller is electrically connected to the display screen, and the controller controls the display screen to display horizontal information about the horizontal data. 7 . The optical sensing device as claimed in claim 4 , wherein the controller calculates the blood flow rate according to the received light signal and records the sensing time or signal extraction time of the received light signal in the memory. 8 . The optical sensing device as claimed in claim 4 , wherein the controller receives and records values related to blood oxygen saturation, systolic pressure or diastolic pressure from the outside in the memory.
9. An optical sensing device, characterized in that: Include: Device housing; An optical sensor is disposed on the housing of the device; a light source disposed on the device housing to project an indication mark in the sensing direction of the optical sensor, and A flexible light shield is disposed on the device housing and surrounds the optical sensor and the light source in the same space. Wherein, the flexible sunshade is a hollow thin shell structure; or The flexible light shield includes a tube sleeve portion and an annular lip portion, wherein the tube sleeve portion is disposed on the device housing, and the annular lip portion extends outward from the tube sleeve portion. The optical sensor has a sensing direction and senses in the sensing direction. The flexible light shield has a cross-sectional profile in the sensing direction, and the cross-sectional profile is circular, elliptical or polygonal.
10. The optical sensing device as described in claim 9 further comprises an image capturing device and a controller, wherein the image capturing device is disposed on the device housing, and the controller is disposed in the device housing and is electrically connected to the optical sensor and the image capturing device, wherein the controller has a memory, and the controller receives optical signals related to blood flow via the optical sensor, and receives and records images in the memory via the image capturing device.
11. The optical sensing device as claimed in claim 10, further comprising an electronic level disposed on the device housing, wherein the controller is electrically connected to the electronic level, and the controller receives and records level data from the electronic level in the memory. 12 . The optical sensing device as claimed in claim 11 , further comprising a display screen exposed from the device housing, wherein the controller is electrically connected to the display screen, and the controller controls the display screen to display level information related to the level data.
13. The optical sensing device as described in claim 10, wherein the light source is electrically connected to the controller, wherein the optical sensor has a sensing direction and senses in the sensing direction, and the controller controls the light source to project the indicator mark in the sensing direction, and the indicator mark is a cross, a circle, an arrow or a polygon. 14 . The optical sensing device as claimed in claim 10 , wherein the controller calculates the blood flow rate according to the received light signal and records the sensing time or signal extraction time of the received light signal in the memory. 15 . The optical sensing device as claimed in claim 10 , wherein the controller receives and records values related to blood oxygen saturation, systolic pressure or diastolic pressure from the outside in the memory.
