Image processing method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide an image processing method, device, electronic device and storage medium, which relate to the field of medical imaging technology; wherein, an image processing method, the method comprising: in response to an opening instruction for a target camera, controlling each camera of a binocular camera to be opened at the same time, displaying a preview window of the camera of the binocular camera through a display unit, controlling each camera of the binocular camera to capture an image, displaying the captured target image in the display unit, and setting a mark point in the target image, and subsequently adjusting the position of the target mark point in the mark point, and based on the image captured by each camera and the current image coordinates of each target mark point, converting the image coordinates of each target mark point, and calculating the size information of the wound, and associating the size information with the target image. It can be seen that this solution can improve the efficiency of detecting the size information of the wound.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] For medical detection scenarios, there is a need to detect the size information of a patient's wound, for example, the need to detect the size information of a patient's pressure sore wound, and the need to detect the size information of a patient's infected area.

[0003] In the related art, when medical staff detect wounds, they usually need to manually measure and record the size information of the wounds. However, manual measurement often takes a lot of time, resulting in low efficiency in measuring the size information of the wounds.

[0004] Therefore, how to improve the efficiency of detecting wound size information is an urgent problem to be solved. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device and storage medium to improve the efficiency of detecting wound size information. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a main controller of an all-in-one device, wherein the all-in-one device is further provided with a binocular camera and a display unit; the method comprises:

[0007] In response to an opening instruction for a target camera, each camera of the binocular camera is controlled to be opened at the same time, and a preview window of the camera of the binocular camera is displayed through the display unit in a predetermined display mode; wherein the target camera is a logical camera obtained by packaging each camera of the binocular camera; the predetermined display mode is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera;

[0008] In response to the target photographing instruction, each camera is controlled to shoot an image to obtain an image shot by each camera; wherein the target photographing instruction is a photographing instruction issued by the user to the preview window of the specified camera when the user needs to determine that a patient's wound is displayed in the sub-area;

[0009] Displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image; wherein the marking points are used to locate the wound;

[0010] In response to a user's position adjustment operation on a target marking point among the plurality of marking points, adjusting the position of the target marking point;

[0011] In response to receiving the size information calculation instruction, based on the images captured by each camera and the current image coordinates of each target marking point, convert the current image coordinates of each target marking point into target coordinates in the world three-dimensional coordinate system;

[0012] Based on the target coordinates of each target marking point, the size information of the wound is calculated, and the size information is associated with the target image.

[0013] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to a main controller of an all-in-one device, wherein the all-in-one device is also provided with a binocular camera and a display unit; the device comprises:

[0014] A first control module, configured to control the cameras of the binocular camera to be turned on simultaneously in response to a start instruction for a target camera, and to display preview windows of the cameras of the binocular camera through the display unit in a predetermined display manner; wherein the target camera is a logical camera obtained by packaging the cameras of the binocular camera; the predetermined display manner is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera;

[0015] A second control module is used to control each camera to capture images in response to a target capture instruction to obtain images captured by each camera; wherein the target capture instruction is a capture instruction issued by the user to the preview window of the specified camera when the user needs to determine whether a patient's wound is displayed in the sub-area;

[0016] A setting module, used for displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image; wherein the marking points are used for locating the wound;

[0017] A position adjustment module, configured to adjust the position of a target marking point among the plurality of marking points in response to a user's position adjustment operation on the target marking point;

[0018] A conversion module, configured to respond to receiving the size information calculation instruction and convert the current image coordinates of each target marking point into target coordinates in a world three-dimensional coordinate system based on the images captured by each camera and the current image coordinates of each target marking point;

[0019] A calculation module is used to calculate the size information of the wound based on the target coordinates of each target marking point, and associate the size information with the target image.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0021] Memory, used to store computer programs;

[0022] The processor is used to implement any of the above-mentioned image processing methods when executing the program stored in the memory.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned image processing methods is implemented.

[0024] Beneficial effects of the embodiments of the present application:

[0025] It can be seen that the image processing method provided in the embodiment of the present application does not require manual measurement when detecting the size information of the wound, does not require a lot of time, greatly saves human resources, and can improve the efficiency of wound detection; and the embodiment of the present application can respond to the start instruction for the target camera and control the various cameras of the binocular camera to start at the same time, which can ensure the synchronization of the images collected by the various cameras of the binocular camera (that is, ensure the imaging synchronization). In theory, the delay between the cameras for imaging can reach 0ms, thereby improving the accuracy of wound detection.

[0026] Furthermore, when calculating the size information of the wound, the present application takes into account the images captured by each camera of the binocular camera, and also ensures the accuracy of the detection of the wound. In addition, the embodiment of the present application can automatically associate the calculated size information of the wound with the target image, and the medical staff can subsequently retrieve the target image without manually associating the size information of the wound with the target image, which reduces the possibility of errors and also improves the efficiency of wound detection.

[0027] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0029] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0030] FIG. 2 (a) is a schematic diagram of an effect of a predetermined display method provided in an embodiment of the present application;

[0031] FIG2( b ) is a schematic diagram of a sub-region highlighting effect provided by an embodiment of the present application;

[0032] FIG2 (c) is a schematic diagram of an effect of prompt information in a preview window provided by an embodiment of the present application;

[0033] FIG2( d ) is a schematic diagram of the effect of each target marking point in a sub-region provided in an embodiment of the present application;

[0034] FIG3 (a) is a schematic diagram of an effect of guidance information provided by an embodiment of the present application;

[0035] FIG3 (b) is another schematic diagram of the effect of guidance information provided by an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the effect of a marking point provided in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of a top-down perspective effect of photographing a physical point with a binocular camera provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of the effect of associating size information with a target image provided in an embodiment of the present application;

[0039] Figure 7 A flowchart of another image processing method provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0041] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0043] First, some professional terms in the embodiments of the present application are briefly introduced:

[0044] Binocular camera calibration file: A reference file for restoring the world's 3D coordinate system based on images. It can also be considered as a reference file for converting image coordinates to coordinates in the world's 3D coordinate system.

[0045] Binocular camera: A camera module with the same specifications and parameters. A binocular camera can contain two cameras.

[0046] Distortion correction: used to restore geometrically distorted images to normal state.

[0047] And, in order to better understand this solution, the following is a brief introduction to the relevant technologies:

[0048] Based on a single camera taking an image containing a wound, image recognition is performed on the image to identify the size information of the wound in a three-dimensional coordinate system. However, in the process of image recognition, it is easily affected by the surrounding environment, resulting in inaccurate size information of the wound. For example, when the angle of shooting the wound is crooked, the size information of the wound may be inaccurate. In addition, only referring to a single image to calculate the size information of the wound will also result in low accuracy in detecting the size information of the wound.

[0049] Based on the problems described above, embodiments of the present application provide an image processing method, device, electronic device, and storage medium.

[0050] The image processing method provided in the embodiment of the present application is introduced below.

[0051] Among them, the image processing method provided in the embodiment of the present application can be applied to the main controller of the all-in-one device, and the all-in-one device can be a device with a binocular camera and a display unit; specifically, the all-in-one device can be various handheld devices, such as: mobile phone devices, tablet computer devices, etc. Of course, the all-in-one device can also be a non-handheld device, and the embodiment of the present application does not make specific limitations on this.

[0052] Furthermore, the binocular camera can be a camera with two cameras, the two cameras can be a left camera and a right camera respectively, and any binocular camera on the market can be used as the binocular camera in the embodiment of the present application. In addition, the main controller of the all-in-one device can also be considered as the central processing unit CPU or core controller of the all-in-one device, and the embodiment of the present application does not specifically limit this.

[0053] The following is a brief introduction to the application scenarios of the image processing method provided in the embodiment of the present application:

[0054] The application scenario of the embodiment of the present application may be a medical photography scenario, and photography may be performed on the patient's wounds, for example: detecting the size information of the patient's pressure sore wounds, detecting the size information of the patient's infected area, and the embodiment of the present application does not make any specific limitations on this.

[0055] Among them, an image processing method is applied to a main controller of an all-in-one device, and the all-in-one device is also provided with a binocular camera and a display unit; the method comprises:

[0056] In response to an opening instruction for a target camera, each camera of the binocular camera is controlled to be opened at the same time, and a preview window of the camera of the binocular camera is displayed through the display unit in a predetermined display mode; wherein the target camera is a logical camera obtained by packaging each camera of the binocular camera; the predetermined display mode is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera;

[0057] In response to the target photographing instruction, each camera is controlled to shoot an image to obtain an image shot by each camera; wherein the target photographing instruction is a photographing instruction issued by the user to the preview window of the specified camera when the user needs to determine that a patient's wound is displayed in the sub-area;

[0058] Displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image; wherein the marking points are used to locate the wound;

[0059] In response to a user's position adjustment operation on a target marking point among the plurality of marking points, adjusting the position of the target marking point;

[0060] In response to receiving the size information calculation instruction, based on the images captured by each camera and the current image coordinates of each target marking point, convert the current image coordinates of each target marking point into target coordinates in the world three-dimensional coordinate system;

[0061] Based on the target coordinates of each target marking point, the size information of the wound is calculated, and the size information is associated with the target image.

[0062] It can be seen that the image processing method provided in the embodiment of the present application does not require manual measurement when detecting the size information of the wound, does not require a lot of time, greatly saves human resources, and can improve the efficiency of wound detection; and the embodiment of the present application can respond to the start instruction for the target camera and control the various cameras of the binocular camera to start at the same time, which can ensure the synchronization of the images collected by the various cameras of the binocular camera (that is, ensure the imaging synchronization). In theory, the delay between the cameras for imaging can reach 0ms, thereby improving the accuracy of wound detection.

[0063] Furthermore, when calculating the size information of the wound, the present application takes into account the images captured by each camera of the binocular camera, and also ensures the accuracy of the detection of the wound. In addition, the embodiment of the present application can automatically associate the calculated size information of the wound with the target image, and the medical staff can subsequently retrieve the target image without manually associating the size information of the wound with the target image, which reduces the possibility of errors and also improves the efficiency of wound detection.

[0064] An image processing method provided by an embodiment of the present application is described below with reference to the accompanying drawings.

[0065] like Figure 1 As shown, an image processing method provided in an embodiment of the present application is applied to a main controller of an all-in-one device, wherein the all-in-one device is also provided with a binocular camera and a display unit; the method comprises:

[0066] S101, in response to an opening instruction for a target camera, controlling each camera of the binocular camera to be opened simultaneously, and displaying a preview window of the camera of the binocular camera through the display unit according to a predetermined display mode;

[0067] The target camera is a logical camera obtained by packaging each camera of the binocular camera; the predetermined display method is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera;

[0068] It can be understood that each sensor sensor on the all-in-one device can be mapped to a sensor ID (Identity document). Then, each camera of the binocular camera set on the all-in-one device can be mapped to a camera ID. Each camera is mapped to a camera ID, and the camera ID can also be called a physical ID or physicalCameraID. Then, each camera of the binocular camera can be encapsulated to obtain a logical camera, that is, a target camera. The physical ID of each camera of the binocular camera can also be mapped to the ID of the logical camera. The ID of the logical camera can also be called a target ID, or logiclCameraID. For example, the camera ID of camera 1 is physicalCameraID 2, and the camera ID of camera 2 is physicalCameraID 4. Camera 1 and camera 2 can be encapsulated to obtain a logical camera, and the ID of the logical camera is logiclCameraID5. In addition, when the all-in-one device is an Android device, the camera device cameraDevice of the Android device can control the target camera to be turned on, thereby controlling the various cameras of the binocular camera to be turned on at the same time, and obtaining the preview images collected by each camera of the binocular camera. The preview images collected by each camera are completely synchronized without any delay.

[0069] For a clear layout, the method of packaging each camera of the binocular camera and the process of controlling each camera of the binocular camera to be turned on at the same time will be introduced in subsequent embodiments and will not be described in detail again.

[0070] Furthermore, after the various cameras of the binocular camera are turned on at the same time, the preview function of the various cameras of the binocular camera can be turned on at the same time, and the preview windows of the cameras of the binocular camera can be displayed through the display unit according to the predetermined display method; the predetermined display method is to display only the preview image captured by one camera of the binocular camera in the display unit, that is, the preview window of the designated camera of the binocular camera is visible to the user, and the preview windows of other cameras of the binocular camera except the designated camera are not visible to the user. It can be understood that the predetermined display method can be to display the preview windows of each camera in an overlapping manner, and only display the top preview window, or only display the preview window of one camera, and hide the preview windows of other cameras. This application does not make specific limitations on this.

[0071] In one implementation, the predetermined display method includes:

[0072] The preview windows of the various cameras are displayed overlappingly, and the preview window of a specified camera is set on an upper layer of the preview window of the other camera.

[0073] It is understandable that the preview windows of each camera can be displayed overlappingly. Then, the preview window of the specified camera needs to be set on the upper layer of the preview window of another camera. Therefore, only the preview window of the specified camera can be displayed, and the preview window of the other camera will not be displayed. The user can only see the preview window of the specified camera on the display unit.

[0074] In order to better understand the relevant content of the scheduled display method, a brief introduction is given below in conjunction with the accompanying drawings, as shown in Figure 2 (a):

[0075] FIG2 (a) is a mobile phone screen, in which the preview windows of the binocular cameras overlap, and the preview window of the left camera is located above the preview window of the right camera. Only the preview window of the left camera is visible, and the preview window of the right camera is not visible. However, the preview window of the right camera also exists.

[0076] It can be seen that the embodiment of the present application can make the preview window of the designated camera of the binocular camera visible to the user through the preview display method, thereby ensuring that the user can only see the preview window of the designated camera, and will not see the preview windows of two cameras at the same time, thereby improving the user experience.

[0077] It can be understood that there is a sub-area with a highlighting effect in the preview window of the specified camera, so that the sub-area is displayed more prominently in the preview window; and the acquisition range corresponding to the sub-area belongs to the overlapping field of view of each camera, that is, there is a sub-area in the overlapping field of view of the two cameras of the binocular camera, and the sub-area has a highlighting effect in the preview window of the specified camera. And, highlighting the sub-area can be to increase the brightness of the sub-area, or to lower the brightness of other areas, and to add a specified shape of edge line to the edge of the sub-area, and highlight the sub-area through the edge line of the specified shape. The embodiment of the present application does not make specific restrictions on this. It should be noted that the sub-area can be used as a sub-preview window of the patient's wound; and since the field of view corresponding to the sub-area belongs to the overlapping video range of each camera, if a wound is displayed in the sub-area, that is, the wound can be previewed through the sub-preview window, it means that the patient's wound is collected by each camera, which is convenient for the subsequent calculation of the size information of the wound based on the images of each camera to ensure the accuracy of the calculation of the size information.

[0078] Exemplarily, in one implementation, the setting method of the highlighting effect of the sub-area of ​​the preview window of the specified camera includes:

[0079] A mask is set for an area outside the sub-area of ​​the preview window of the specified camera to highlight the sub-area.

[0080] It is understandable that a mask is set for the area outside the sub-area of ​​the preview window of the specified camera. The mask can be considered as a layer with lower brightness, while the sub-area is transparent, thereby achieving the effect of highlighting the sub-area.

[0081] In order to better understand the relevant contents about the highlighting effect of the sub-area of ​​the preview window of the specified camera, a brief introduction is given below in conjunction with the accompanying drawings, as shown in FIG2(b):

[0082] It can be understood that the preview window 210 in FIG. 2 ( b ) may include a sub-area 211 and other areas 212 , where the other areas 212 are areas other than the sub-areas of the preview window described above, the sub-area 211 has a highlighting effect, and the other areas 212 are provided with a mask layer, and the brightness of the mask layer is relatively low.

[0083] It can be seen that the embodiment of the present application can set a mask for the area outside the sub-area of ​​the preview window of the specified camera to highlight the sub-area, which can facilitate users to identify the sub-area and improve the user experience.

[0084] In one implementation, prompt information is displayed in the preview window of the designated camera; wherein the prompt information is used to instruct the user to control the wound to be displayed within the sub-area.

[0085] It is understandable that prompt information can also be displayed in the preview window of the specified camera, and the prompt information can be used to instruct the user to control the wound to be displayed in the sub-area, so as to facilitate subsequent photo taking. Only when the wound is displayed in the sub-area can the binocular camera be controlled to take photos.

[0086] Of course, other information may also be set in the preview window of the specified camera, such as current time information, current brightness information, etc., and the embodiments of the present application do not specifically limit this.

[0087] In order to better understand the relevant content of the prompt information displayed in the preview window of the specified camera, a brief introduction is given below in conjunction with the accompanying drawings, as shown in Figure 2 (c):

[0088] It is understandable that the preview window 210 may include sub-areas 211 and other areas 212. The highlighted area in FIG. 2 (c) may be considered as the sub-area 211 described above. In this figure, the wound is a sore, so the prompt message may be: Please keep the sore within the highlighted area.

[0089] It can be seen that prompt information can be displayed in the preview window of a fixed camera in the embodiment of the present application, so as to prompt the user to control the wound to be displayed in the sub-area, facilitate the user to take pictures later, and improve the user experience.

[0090] S102, in response to the target photographing instruction, controlling each camera to photograph an image, and obtaining an image photographed by each camera;

[0091] The target photo-taking instruction is a photo-taking instruction issued by the user to the preview window of the designated camera when the user needs to confirm that a patient's wound is displayed in the sub-area;

[0092] It is understandable that, based on the prompt information introduced above, when the user determines that a patient's wound is displayed in the sub-area, the user can issue a photo command for the preview window of the specified camera, that is, issue a target photo command; in response to the target photo command, each camera can be controlled to capture images to obtain images captured by each camera; it should be emphasized that it is not that the specified camera is taken alone, but that each camera of the binocular camera is controlled to capture images, so as to obtain images captured by each camera. In addition, after obtaining the images captured by each camera, the images captured by each camera can be corrected for distortion, so that the distorted images are restored to a normal state. It should be noted that any implementation method for distortion correction in the relevant technology can be applied to the present application.

[0093] S103, displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image;

[0094] Wherein, the marking point is used to locate the wound;

[0095] It is understandable that after obtaining the images captured by each camera, the target image captured by the designated camera can be displayed on the display unit, and a mark point can be set in the target image data, and the mark point can be a point used to preliminarily locate the wound. Also, the image captured by another camera of the binocular camera may not be displayed on the display unit, and the image captured by another camera may be saved in a designated server for use in subsequent steps; wherein the designated server may be a server for storing patient information and images captured by the camera.

[0096] For the sake of clarity, the process of setting marking points at multiple initial positions in the target image will be introduced in subsequent embodiments and will not be described in detail again.

[0097] S104, in response to a user's position adjustment operation on a target marking point among the multiple marking points, adjusting the position of the target marking point;

[0098] Among them, there may be a target marking point among the multiple marking points, and the number of target marking points is multiple. The target marking point is the marking point indicated by the position adjustment operation, that is, the marking point adjusted by the user; it should be emphasized that all of the multiple marking points can be used as target marking points, or only at least two marking points can be used as target marking points, and the embodiments of the present application do not specifically limit this. It is understandable that when the target image is displayed on the display unit, the user can adjust the position of the target marking point among the marking points on the display unit, so that the position of the target marking point can be adjusted in response to the position adjustment operation.

[0099] For the sake of clarity of the layout, in response to the user's position adjustment operation on the target marking point among the multiple marking points, the process of adjusting the position of the target marking point will be introduced in subsequent embodiments and will not be described in detail again.

[0100] S105, in response to receiving the size information calculation instruction, based on the images captured by each camera and the current image coordinates of each target marking point, convert the current image coordinates of each target marking point into target coordinates in the world three-dimensional coordinate system;

[0101] The current image coordinates of each target marking point are the image coordinates of the target marking point after the position of the target marking point is adjusted in response to the position adjustment operation;

[0102] It is understandable that after the position of the target marker point is adjusted, the user, such as a relevant medical staff, can issue a size information calculation instruction. For example, the size calculation instruction can be issued by touching the button element in the preview window, or triggered by voice; accordingly, in response to receiving the size information calculation instruction, the current image coordinates of each target marker point can be converted into the target coordinates in the world three-dimensional coordinate system, and the current image coordinates of each target marker point are the image coordinates after the position adjustment for each target marker point; and when converting the current image coordinates of each target marker point into the target coordinates in the world three-dimensional coordinate system, the images taken by each camera can be referred to, rather than only the images taken by the specified camera. More reference dimensions also improve the accuracy of wound detection. In addition, the current image coordinates of the target marker point can be determined based on the pixels of the target image. For example, the pixels of the target image are 2448×3264, the image coordinates of the target marker point 1 are (1200, 1000), and the image coordinates of the target marker point 2 are (1600, 1000).

[0103] It should be emphasized that after receiving the size information calculation instruction, it is not necessary to immediately execute the step of converting the current image coordinates of each target marker point into the target coordinates in the world three-dimensional coordinate system based on the images taken by each camera and the current image coordinates of each target marker point. In this process, there may also be a certain delay. For example, the coordinate conversion step is executed after a delay of 10ms. The embodiment of the present application does not make specific limitations on this.

[0104] For the sake of clarity, the process of converting the current image coordinates of each target marker point into the target coordinates in the world three-dimensional coordinate system will be introduced in subsequent embodiments and will not be described in detail again.

[0105] S106, calculating the size information of the wound based on the target coordinates of each target marking point, and associating the size information with the target image.

[0106] It can be understood that the target coordinates are coordinates in the world three-dimensional coordinate system. Therefore, the distance calculation formula and the target coordinates of each target marking point can be used to calculate the actual size information of the wound, that is, the size information of the wound in the world three-dimensional coordinate system; wherein, the size information of the wound may at least include the length and width of the wound. Of course, this is only an exemplary introduction here, and the size information may also be the length of the diagonal line, and this application does not make any specific limitations on this.

[0107] Exemplarily, the distance calculation formula is: ;

[0108] It should be emphasized that the distance calculation formula is applicable to the scenario where each target marking point belongs to multiple groups. In this scenario, each group contains two target marking points, and the two target marking points in each group can be used to indicate the size information of the wound; specifically, the two target marking points in one group can be used to indicate the length of the wound, and the two target marking points in the other group can be used to indicate the width of the wound. Substituting the target marking points of the above two groups into the above distance calculation formula, the length and width can be calculated. The above is only an exemplary introduction, and the embodiments of the present application do not make specific limitations on this.

[0109] Among them, the target coordinates of a target mark point in any group are ( ), the target coordinates of another target marker in this group are ( ), distance represents the size information of the wound, and the above formula can be considered as a formula for calculating the distance based on two three-dimensional coordinates. Of course, using the distance calculation formula to calculate the size information of the wound is only an exemplary introduction, and other methods can also be used to calculate the size information of the wound, and the embodiments of the present application do not specifically limit this.

[0110] In order to better understand this part of the content, the following is an introduction with reference to the accompanying drawings, as shown in Figure 2 (d):

[0111] FIG2 (d) is a sub-area 211 of the preview window. There are four target marking points in the figure, among which point 1 and point 2 belong to the same group, and point 3 and point 4 belong to the same group. Substituting the three-dimensional coordinates of point 1 and point 2 into the above formula can calculate the width of the wound, and substituting the three-dimensional coordinates of point 3 and point 4 into the above formula can calculate the length of the wound.

[0112] In addition, associating the wound size information with the target image can be considered as adding the size information to the target image, and this application does not make any specific limitation to this.

[0113] It can be seen that the image processing method provided in the embodiment of the present application does not require manual measurement when detecting the size information of the wound, does not require a lot of time, greatly saves human resources, and can improve the efficiency of wound detection; and the embodiment of the present application can respond to the start instruction for the target camera and control the various cameras of the binocular camera to start at the same time, which can ensure the synchronization of the images collected by the various cameras of the binocular camera (that is, ensure the imaging synchronization). In theory, the delay between the cameras for imaging can reach 0ms, thereby improving the accuracy of wound detection.

[0114] Furthermore, when calculating the size information of the wound, the present application takes into account the images captured by each camera of the binocular camera, and also ensures the accuracy of the detection of the wound. In addition, the embodiment of the present application can automatically associate the calculated size information of the wound with the target image, and the medical staff can subsequently retrieve the target image without manually associating the size information of the wound with the target image, which reduces the possibility of errors and also improves the efficiency of wound detection.

[0115] Optionally, in another embodiment of the present application, the method of packaging each camera of the binocular camera includes method A1:

[0116] Method A1, establishing a mapping relationship between the physical ID of each camera and the target ID; wherein the target ID is the ID of the target camera;

[0117] Accordingly, in response to the start instruction for the target camera, each camera of the binocular camera is controlled to be turned on simultaneously, including step B1:

[0118] Step B1, in response to an on instruction for a target camera, determining the physical ID to which the target ID of the target camera is mapped from the mapping relationship, and controlling each camera with the determined physical ID to be turned on simultaneously.

[0119] It can be understood that, for method A1, the physical ID of each camera can also be called camera ID or physicalCameraID, and the target ID can also be called logical camera ID or logiclCameraID, so a mapping relationship can be established between the physical ID of each camera of the binocular camera and the ID of the logical camera obtained after pre-packaging; exemplarily, the physical ID of camera 1 is physicalCameraID 2, the physical ID of camera 2 is physicalCameraID 4, and the target ID is logiclCameraID5, so a mapping relationship can be established between physicalCameraID 2, physicalCameraID 4 and logiclCameraID5.

[0120] It can be understood that, for step B1, the all-in-one device can be turned on by controlling the target camera, responding to the turn-on instruction of the target camera, determining the physical ID to which the target ID of the target camera is mapped based on the mapping relationship between the physical ID of each camera and the target ID, and controlling each camera with the physical ID to be turned on at the same time based on the physical ID, thereby realizing that each camera of the binocular camera is turned on at the same time, and there is a 0ms delay between each camera of the binocular camera in this process, thereby ensuring the accuracy of wound detection. Exemplarily, in response to the turn-on instruction of the turn-on instruction of the target camera, determining the physical ID physicalCameraID 2 and physicalCameraID 4 to which the target ID logiclCameraID5 of the target camera is mapped, and controlling camera 1 and camera 2 with physical ID physicalCameraID 2 and physicalCameraID 4 to be turned on at the same time.

[0121] It can be seen that the embodiment of the present application can pre-establish a mapping relationship between the physical ID of each camera and the target ID. When responding to an on instruction for the target camera, the physical ID to which the target ID of the target camera is mapped can be determined from the mapping relationship, and each camera with a determined physical ID can be controlled to be turned on at the same time, so that the delay between each camera of the binocular camera is 0, which can ensure the real-time nature of the images captured by the binocular camera, thereby improving the accuracy of wound detection.

[0122] Optionally, in another embodiment of the present application, the shape of the sub-region is a specified graphic shape;

[0123] The determination method of the multiple initial positions includes method C1:

[0124] Mode C1, determining multiple initial positions according to a position determination method that matches the shape category of the specified graphic shape.

[0125] The specified graphic shape may be a rectangle, a circle, etc., and this embodiment of the present application does not specifically limit this.

[0126] It is understandable that when the shape of the sub-region is a specified image shape, multiple initial positions can be determined according to a position determination method that matches the shape category of the specified graphic shape; for example, when the sub-region is a circle, multiple positions on the circle that are connected and can represent the diameter can be determined as the initial position. And, depending on the specified graphic shape, the position determination method that matches the corresponding shape category is also different, and the embodiments of the present application do not specifically limit this.

[0127] Of course, when the shape of the sub-region is not a specified graphic shape, any multiple positions can also be determined in the sub-region as the initial positions, and the embodiment of the present application does not make any specific limitations on this.

[0128] It can be seen that the embodiment of the present application can determine multiple initial positions according to a position determination method that matches the shape category of the specified graphic shape, and then mark points can be set for the multiple initial positions, so that the initial positions can be quickly located, thereby improving the efficiency of detecting the size information of the wound.

[0129] Exemplarily, in one implementation, the specified graphic shape is a rectangle;

[0130] Correspondingly, in mode C1, the multiple initial positions are determined according to the position determination mode matching the shape category of the specified graphic shape, including mode C11:

[0131] Method C11, determining the midpoint position of each side of the border of the sub-region to obtain multiple initial positions.

[0132] It can be understood that method C11 is for the case where the shape of the specified graphic is a rectangle. Then, the position determination method that matches the shape category of the rectangle is: determine the midpoint position of each side of the rectangle represented by the sub-region, thereby determining multiple initial positions, and then a mark point can be set at each initial position. Exemplarily, the shape of the sub-region is a rectangle, the long border of the sub-region is 8cm, and the wide border is 6cm. Then, the midpoint of the long border and the midpoint of the wide border of the sub-region can be determined to obtain multiple initial positions, that is, the position of the long border of 4cm and the position of the wide border of 3cm. Of course, the above is only an exemplary introduction, and this corner of the rectangle can also be determined as multiple initial positions, and the embodiment of the present application does not make specific limitations on this.

[0133] It can be seen that when the specified graphic shape is a rectangle, the midpoint position of each side of the border of the sub-area can be determined to obtain multiple initial positions, so that the initial position can be quickly located, thereby improving the efficiency of detecting the size information of the wound.

[0134] Optionally, in another embodiment of the present application, the all-in-one device is further provided with a level;

[0135] In response to the target photographing instruction, each camera is controlled to photograph an image, and before the image photographed by each camera is obtained, the method further includes step D1:

[0136] Step D1, start the level and display guidance information about whether the device is in a horizontal posture identified by the level in the preview window of the designated camera, so as to guide the user to adjust the posture of the all-in-one device to a horizontal posture.

[0137] Among them, the level is used to identify whether the all-in-one device is in a horizontal posture;

[0138] It is understandable that before the user sends a target photo-taking instruction, it is necessary to first start the level meter set in the all-in-one device. The level meter can identify whether the all-in-one device is in a horizontal posture, and display guidance information in the preview window to guide the user to adjust the posture of the all-in-one device to a horizontal posture; then, if the level meter recognizes that the all-in-one device is in a horizontal posture, the displayed guidance information is specifically to guide the user to send a target photo-taking instruction, and vice versa, if the level meter recognizes that the all-in-one device is not in a horizontal posture, the displayed guidance information is specifically to guide the user to adjust the posture of the all-in-one device to a horizontal posture. Exemplarily, the level meter is started, and the level meter detects that the mobile phone device is not in a horizontal posture. At this time, the guidance information that the mobile phone device is not in a horizontal posture identified by the level meter can be displayed to guide the user to adjust the posture of the mobile phone device to a horizontal posture.

[0139] Of course, the horizontal posture is not limited to being completely horizontal, that is, the shooting angle of the all-in-one device relative to the wound is 0°, and the allowed shooting angle is between 5° and -5°. The embodiment of the present application does not make specific limitations on this.

[0140] In order to better understand the content of the guidance information, the following is an introduction with reference to the accompanying drawings, as shown in FIG3 (a) and FIG3 (b):

[0141] FIG3 (a) shows that the mobile phone device is not in a horizontal posture, as recognized by the level meter. The preview window 210 may include a sub-area 211 and other areas 212. At this time, the circle with a shadow in the preview window 210 is not aligned with the central circle of the preview window 210. At this time, the user can be guided to adjust the posture of the mobile phone device to a horizontal posture, and a prompt message "Please control the wound within the highlighted area" is displayed at the lower left corner of the preview window 210. There are also a camera button and a flashlight start button at the bottom of the preview window 210. There is also a display title of "Binocular Distance Measurement" and a "?" button, and a "Gear" button representing the setting at the top of the preview window 210.

[0142] FIG3 (b) shows that the mobile phone device is in a horizontal position as recognized by the level meter. The preview window 210 may include a sub-area 211 and other areas 212. At this time, the circle with a shadow in the preview window 210 is aligned with the central circle of the preview window 210. At this time, the user can be guided to control the binocular camera to shoot, and a prompt message "Please control the wound within the highlighted area" is displayed at the lower left corner of the preview window 210. There are also a photo button and a flashlight start button at the bottom of the preview window 210. There is also a display title of "Binocular Distance Measurement" and a "?" button, and a "Gear" button representing the setting at the top of the preview window 210.

[0143] It can be seen that the embodiment of the present application can utilize the spirit level provided in the all-in-one device to display in the preview window guidance information on whether the device is in a horizontal posture as identified by the spirit level, thereby guiding the user to adjust the posture of the all-in-one device to a horizontal posture. Only when the posture of the all-in-one device is a horizontal posture can each camera be controlled to capture images, thereby ensuring the accuracy of wound detection.

[0144] Optionally, in another embodiment of the present application, each marking point belongs to a plurality of groups, each group includes two marking points, and different groups include different marking points;

[0145] In response to a user's position adjustment operation on a target marking point among the multiple marking points, adjusting the position of the target marking point may include step E1:

[0146] Step E1, in response to a user's position adjustment operation on a target mark point among the multiple mark points, displaying a display effect of the target mark point according to a target display mode, and adjusting the position of the target mark point according to an adjustment track indicated by the position adjustment operation;

[0147] The target display method includes enlarging and displaying the target mark point, and the target mark point and another mark point in the group to which the target mark point belongs have a connecting line.

[0148] It can be understood that when marking points are set at multiple initial positions in the target image, the marking points have been grouped, and each marking point belongs to multiple groups, each group contains two marking points, different groups contain different marking points, and the two marking points in each group can represent the size information of the wound in the image; then, at least 4 marking points need to be set in the target image to represent the length and width of the wound in the image. Exemplarily, marking point 1 and marking point 2 belong to one group, and marking point 3 and marking point 4 belong to one group, wherein marking point 1 and marking point 2 can represent the length of the wound in the image, and marking point 3 and marking point 4 can represent the width of the wound in the image.

[0149] It can be understood that, in response to the user's position adjustment operation on a target marking point among multiple marking points, the display effect of the target marking point can be displayed according to the target display mode. In the target display mode, the target marking point can be enlarged and displayed, and the target marking point and another marking point in the group to which the target marking point belongs have a connecting line, which can be a solid line; based on the target display mode, the user can clearly see the position of the target marking point, and the connecting line between the target marking point and another marking point in the group to which the target marking point belongs.

[0150] Furthermore, the position of the target marker point can be adjusted according to the adjustment track indicated by the position adjustment operation so that the current position of the target marker point is the position where the user expects the target marker point to be located; the current position of the target marker point can represent the size information of the wound in the target image.

[0151] For a better understanding, the contents of each mark point belonging to multiple groups are introduced below with reference to the accompanying drawings. Figure 4 As shown:

[0152] Target marker point 1 and target marker point 2 belong to a group and are used to indicate the length of the wound. Target marker point 3 and target marker point 4 belong to a group and are used to indicate the width of the wound. The pressure sore staging option is stage IV pressure sore. The measurement site is the elbow (left). "Pressure Sore Size" is displayed above the preview window, and a "Start Calculation" button is displayed below the preview window, indicating that the size information of the pressure sore can be calculated.

[0153] It can be seen that the embodiment of the present application can respond to the user's position adjustment operation on the target marking point among multiple marking points, display the display effect of the target marking point according to the target display method, and adjust the position of the target marking point according to the adjustment trajectory indicated by the position adjustment operation. Displaying it according to the target display method can enable the user to see the target point and another marking point in the group to which the target point belongs more clearly, thereby improving the user experience; and the user can also adjust the position of the target marking point, thereby ensuring the accuracy of wound detection.

[0154] Optionally, in another embodiment of the present application, the converting of the current image coordinates of each target marking point into the target coordinates in the world three-dimensional coordinate system based on the images captured by each camera and the current image coordinates of each target marking point includes steps F1-F2:

[0155] Step F1, determining the marking points corresponding to each target marking point in the image captured by another camera other than the designated camera, to obtain each designated marking point;

[0156] It is understandable that the two cameras of the binocular camera can simultaneously capture images of the wound, and there is a complete wound within the overlapping field of view of the two cameras, so there is a complete wound in the target image captured by the designated camera and the image captured by another camera other than the designated camera; then, the pixel conversion relationship between the two cameras of the binocular camera can be established in advance, so that after knowing the coordinates of each target marking point in the target image captured by the designated camera, the marking point corresponding to the target marking point in the image captured by the other camera can be determined based on the above pixel conversion relationship, that is, each designated marking point. Of course, it is also possible to obtain the image coordinates of each target marking point in the designated camera, and determine the marking points corresponding to each target marking point in the image captured by another camera other than the designated camera through an image matching algorithm. Specifically, any image matching algorithm can be used as the image matching algorithm in this application, and no specific limitation is made here; and this application does not limit the specific implementation method of determining the marking points corresponding to each target marking point.

[0157] Step F2, based on the current image coordinates of each target marking point, the current image coordinates of each designated marking point and the reference file, convert the current image coordinates of each target marking point into the target coordinates in the world three-dimensional coordinate system; wherein the reference file is a file used to convert the image coordinates into coordinates in the world three-dimensional coordinate system.

[0158] It can be understood that the reference file is a pre-established auxiliary file for converting image coordinates into coordinates in the world three-dimensional coordinate system. The reference file can be used to facilitate the conversion of image coordinates; then, the current image coordinates of each target marker point can be determined first. When converting the current image coordinates of each target marker point, the current image coordinates of each designated marker point can be taken into account, rather than just the image taken by the designated camera. In this way, the target coordinates in the world three-dimensional coordinate system converted from the images taken by the two cameras of the binocular camera will be more accurate. In addition, the reference file can also be called a binocular camera calibration file, which is not specifically limited in the embodiments of the present application. And, any coordinate conversion method can be applied to the embodiments of the present application, which is not specifically limited in the present application.

[0159] For example, in one implementation, the principle of converting the current image coordinates of each target marker point into the target coordinates in the world three-dimensional coordinate system can be found in Figure 5 As shown:

[0160] Figure 5 The top view of the binocular camera taken at the physical point O, wherein the binocular camera may include a left camera and a right camera. In this embodiment, the left camera is the designated camera introduced in the above embodiment, and the left camera may also be referred to as CL, that is, Camera Left, the right camera can also be called CR, that is, CameraRight, the distance between the left camera and the right camera is b, which can also be called the baseline distance. The baseline distance can be obtained through the above reference file, the point corresponding to the physical point O in the image taken by the left camera is p, and the point corresponding to the physical point O in the image taken by the right camera is p', ZL is the Z axis corresponding to the left camera, ZR is the Z axis corresponding to the right camera, the vertical distance between the left camera and the right camera from the X axis is the focal length f, the vertical distance between the left camera and the right camera from the physical point O in the direction of the Z axis is Z, the vertical distance between the left camera and the physical point O in the direction of the X axis is X, the vertical distance between the left camera and p in the direction of the X axis is xL, and the vertical distance between the right camera and p' in the direction of the X axis is xR; it should be emphasized that the parallax between the left camera and the right camera can be considered as the difference between xL and xR; and p and p' can be called co-points.

[0161] And, based on Figure 5 The top-down perspective can be used to calculate the distance (X) of the physical point O about the X axis and the distance (Z) about the Z axis in the world three-dimensional coordinate system; specifically, for the left camera, based on the principle of similar triangles, the formula can be obtained: , through transformation, we can get Z=X×f / xL; for the right camera, we can also get the formula based on the principle of similar triangles: , through transformation we can get Z=(Xb)×f / xR.

[0162] In addition, based on the camera coordinate system of the right camera, the three-dimensional coordinates of p' in the camera coordinate system of the right camera can be obtained. Through the y value of the three-dimensional coordinates, the distance (Y) of the physical point O about the Y axis in the world three-dimensional coordinate system can be obtained. Specifically, through the three-dimensional coordinates of p' in the camera coordinate system of the right camera, the Pythagorean theorem can be used to calculate the distance of p' from the origin of the camera coordinate system of the right camera, and p' and the physical point O are points on the same line. Therefore, through the principle of similar triangles, the distance of the physical point O from the origin of the camera coordinate system of the right camera can also be calculated, so that the distance (Y) of the physical point O about the Y axis in the world three-dimensional coordinate system can be calculated, and then the coordinates of the physical point O in the world three-dimensional coordinate system can be determined.

[0163] It should be emphasized that any coordinate transformation method can be applied to the transformation of points in the image coordinate system to the camera coordinate system in this application, and this application does not make any specific limitations on this; and any method of calculating physical points in the world three-dimensional coordinate system can be applied to this application, and this application does not make any specific limitations on this.

[0164] It can be seen that when the embodiment of the present application converts the current image coordinates of each target marking point into the target coordinates in the world three-dimensional coordinate system, the images collected by each camera of the binocular camera are taken into consideration, thereby ensuring the accuracy of wound detection.

[0165] Optionally, in another embodiment of the present application, the all-in-one device further includes a QR code scanning unit;

[0166] The method further comprises steps G1-G2:

[0167] Step G1, scanning the patient's QR code by a QR code scanning unit, and obtaining patient information based on the QR code;

[0168] It can be understood that the QR code scanning unit can be integrated into a separate scanning camera, which can be set on the top of the all-in-one device. There is a corresponding built-in service inside the all-in-one device. After pressing the camera button, the scanning camera can read the QR code and obtain the patient information through algorithm analysis; then, the patient's QR code can be scanned by the two-dimensional scanning unit to obtain the patient information. The patient information includes but is not limited to the patient's age, gender, admission time, measurement location and measurement time, etc. When the wound type is a sore, the patient information can also include pressure sore staging, where the pressure sore staging can be specifically divided into stage I pressure sores-stage IV pressure sores; patient information can also be stored in the form of documents in a designated server, and the implementation of this application does not specifically limit the patient information.

[0169] Step G2: Binding the patient information with the target image associated with the size information.

[0170] It can be understood that the patient information obtained by the QR code scanning unit can be bound to the target image associated with the size information, and the target image can be considered as a part of the patient information. The patient information can be subsequently saved to the designated server; the next time the patient's QR code is scanned by the QR code scanning unit, the obtained patient information can include the target image associated with the size information.

[0171] It can be seen that the all-in-one device of the embodiment of the present application may also include a QR code scanning unit. After scanning the patient's QR code, the patient information can be obtained based on the QR code, and the patient information can be bound to the target image associated with the size information. The user does not need to manually bind the patient information to the target image, which reduces the possibility of errors and also improves the efficiency of wound detection.

[0172] Optionally, in another embodiment of the present application, associating the size information with the target image may include step H1:

[0173] Step H1: adding the size information as watermark information to the target image, and / or adding the size information to the image name of the target image.

[0174] It can be understood that the specific process of associating the size information with the target image is: adding the size information as watermark information to the target image, and subsequent users can directly see the size information of the wound as watermark information in the target image; the size information can also be directly added to the image name of the target image, for example: the name of the target image is the wound image of patient a (the length of the wound is 30 mm and the width is 40 mm); of course, the size information can also be added to the target image as watermark information, and the size information can also be added to the image name of the target image, so as to achieve the association between the size information and the target image, and the embodiments of the present application do not specifically limit this.

[0175] In order to better understand the relevant content of associating the size information with the target image, the following is introduced in conjunction with the accompanying drawings. Figure 6 As shown:

[0176] The watermark information in the figure indicates that the distance between target mark point 1 and target mark point 2 is 24.1mm, that is, the distance (1-2)-24.1mm, the distance between target mark point 3 and target mark point 4 is 49.95mm, that is, the distance (3-4)-49.95mm, the location: elbow (left), the pressure sore stage: IV pressure sore. In addition, a "Take another photo" button is set under the interface for continuing to take photos, and there is also a title "Pressure Sore Size" at the top of the interface.

[0177] It can be seen that the embodiments of the present application can add the size information as watermark information to the target image, and / or add the size information to the image name of the target image, thereby realizing the association between the size information and the target image. Medical staff can subsequently retrieve the target image without manually associating the wound size information with the target image, thereby reducing the possibility of errors and improving the efficiency of wound detection.

[0178] Optionally, in another embodiment, the present application also provides another image processing method, such as Figure 7 As shown:

[0179] S701, controlling two cameras of the binocular camera to preview simultaneously;

[0180] It can be understood that the two cameras of the binocular camera are controlled to be turned on at the same time, and the two cameras of the binocular camera are controlled to preview at the same time, and the preview pictures are displayed on the display unit.

[0181] S702, starting the level of the mobile phone device;

[0182] It is understandable that when the level on the mobile phone device is started and the level recognizes that the posture of the mobile phone device is horizontal, the user can be prompted to control the binocular camera to collect images.

[0183] S703, performing distortion correction on the image captured by the binocular camera;

[0184] It is understandable that after the image is captured by the binocular camera, distortion correction can be performed on the captured image so that the distorted image is restored to a normal state.

[0185] S704, after distortion correction, automatically generating four points in the wound area in the image captured by the left camera;

[0186] The wound area can be considered as the sub-area introduced in the above embodiment. The wound area is located in the overlapping field of view of the binocular camera. In this embodiment, the wound type is the wound type.

[0187] It is understandable that after the distortion correction, four points can be automatically generated for the sore area in the image captured by the left camera in the binocular camera, and the four points can be used for preliminary positioning of the sore.

[0188] S705, in response to the position adjustment operation, adjusting the position of the generated point;

[0189] It can be understood that, in response to a position adjustment operation on a point in the wound area in the image captured by the left-eye camera, the position of the generated point can be adjusted.

[0190] S706, based on the images captured by each camera of the binocular camera and the image coordinates of the point after the position adjustment, convert the image coordinates of each point after the position adjustment into the target coordinates in the world three-dimensional coordinate system;

[0191] It can be understood that based on the images captured by each camera of the binocular camera and the image coordinates of the points after the position adjustment, the image coordinates of each point after the position adjustment can be converted to obtain the target coordinates in the world three-dimensional coordinate system.

[0192] S707, calculating size information of the wound based on the target coordinates, and associating the size information of the wound with the target image;

[0193] It can be understood that the method of calculating the size information of the wound is roughly the same as the method introduced in the above embodiment, so it will not be described in detail here; and the error of the size information of the wound calculated by this solution can be controlled between 0.5mm-1mm.

[0194] It can be seen that the image processing method provided by the embodiment of the present application does not require manual measurement when detecting the size information of the wound, does not require a lot of time, greatly saves human resources, and can improve the efficiency of wound detection; and, when calculating the size information of the wound, the present application takes into account the images collected by each camera of the binocular camera, and also ensures the accuracy of wound detection. In addition, the embodiment of the present application can automatically associate the calculated size information of the wound with the target image, and the medical staff can subsequently retrieve the target image without manually associating the size information of the wound with the target image, which reduces the possibility of errors and also improves the efficiency of wound detection.

[0195] Based on the above method embodiment, the present application also provides an image processing device, which is applied to the main controller of the all-in-one device, and the all-in-one device is also provided with a binocular camera and a display unit; Figure 8 As shown, the device comprises:

[0196] The first control module 810 is used to control the cameras of the binocular camera to be turned on simultaneously in response to an on instruction for a target camera, and to display preview windows of the cameras of the binocular camera through the display unit in a predetermined display mode; wherein the target camera is a logical camera obtained by packaging the cameras of the binocular camera; the predetermined display mode is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera;

[0197] The second control module 820 is used to control each camera to capture an image in response to a target capture instruction to obtain an image captured by each camera; wherein the target capture instruction is a capture instruction issued by the user to the preview window of the specified camera when the user needs to determine whether a patient's wound is displayed in the sub-area;

[0198] A setting module 830 is used to display the target image captured by the designated camera in the display unit and set marking points at multiple initial positions in the target image; wherein the marking points are used to locate the wound;

[0199] A position adjustment module 840 is used to adjust the position of a target marking point among the multiple marking points in response to a user's position adjustment operation on the target marking point;

[0200] The conversion module 850 is used for responding to receiving the size information calculation instruction, based on the images captured by each camera and the current image coordinates of each target mark point, converting the current image coordinates of each target mark point into the target coordinates in the world three-dimensional coordinate system;

[0201] The calculation module 860 is used to calculate the size information of the wound based on the target coordinates of each target marking point, and associate the size information with the target image.

[0202] Optionally, the method of packaging each camera of the binocular camera includes:

[0203] Establish a mapping relationship between the physical ID of each camera and the target ID; wherein the target ID is the ID of the target camera;

[0204] The first control module comprises:

[0205] The control submodule is used to respond to the start instruction for the target camera, determine the physical ID to which the target ID of the target camera is mapped from the mapping relationship, and control each camera with the determined physical ID to start at the same time.

[0206] Optionally, the setting method of the highlighting effect of the sub-area of ​​the preview window of the specified camera includes:

[0207] A mask is set for an area outside the sub-area of ​​the preview window of the specified camera to highlight the sub-area.

[0208] Optionally, the shape of the sub-region is a specified graphic shape;

[0209] The determination method of the multiple initial positions includes:

[0210] A plurality of initial positions are determined according to a position determination method that matches the shape category of the specified graphic shape.

[0211] Optionally, the specified graphic shape is a rectangle;

[0212] Accordingly, the multiple initial positions are determined according to the position determination method that matches the shape category of the specified graphic shape, including:

[0213] The midpoint position of each side of the border of the sub-region is determined to obtain multiple initial positions.

[0214] Optionally, the all-in-one device is further provided with a level;

[0215] The device also includes:

[0216] The display module is used to control each camera to capture images in response to a target photo instruction, start a level before obtaining the images captured by each camera, and display guidance information about whether the device is in a horizontal posture recognized by the level in the preview window of the specified camera, so as to guide the user to adjust the posture of the all-in-one device to a horizontal posture.

[0217] Optionally, each marking point belongs to a plurality of groups, each group contains two marking points, and different groups contain different marking points;

[0218] The position adjustment module is specifically used for:

[0219] In response to a position adjustment operation of a user on a target mark point among the multiple mark points, displaying a display effect of the target mark point according to a target display mode, and adjusting the position of the target mark point according to an adjustment track indicated by the position adjustment operation;

[0220] The target display method includes enlarging and displaying the target mark point, and the target mark point and another mark point in the group to which the target mark point belongs have a connecting line.

[0221] Optionally, the conversion module is specifically used to:

[0222] Determine the marking points corresponding to the target marking points in the image captured by another camera other than the designated camera, and obtain the designated marking points;

[0223] Based on the current image coordinates of each target marking point, the current image coordinates of each designated marking point and the reference file, the current image coordinates of each target marking point are converted into target coordinates in the world three-dimensional coordinate system; wherein the reference file is a file used to convert image coordinates into coordinates in the world three-dimensional coordinate system.

[0224] Optionally, the all-in-one device further includes a QR code scanning unit;

[0225] The device also includes:

[0226] An acquisition module, used for scanning a patient's QR code through a QR code scanning unit, and acquiring patient information based on the QR code;

[0227] A binding module is used to bind the patient information with a target image associated with the size information.

[0228] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0229] The present application also provides an electronic device, such as Fig. 9 As shown, including:

[0230] Memory 901, used for storing computer programs;

[0231] The processor 902 is used to implement the above-mentioned image processing method when executing the program stored in the memory 901.

[0232] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 902, the communication interface, and the memory 901 communicate with each other via the communication bus.

[0233] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0234] The communication interface is used for communication between the above electronic device and other devices.

[0235] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0236] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0237] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned image processing methods is implemented.

[0238] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any image processing method in the above embodiments.

[0239] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0240] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0241] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0242] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: A main controller applied to an all-in-one device, wherein the all-in-one device is further provided with a binocular camera and a display unit; the method comprises: In response to an opening instruction for a target camera, each camera of the binocular camera is controlled to be opened at the same time, and a preview window of the camera of the binocular camera is displayed through the display unit in a predetermined display mode; wherein the target camera is a logical camera obtained by packaging each camera of the binocular camera; the predetermined display mode is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera; In response to the target photographing instruction, each camera is controlled to shoot an image to obtain an image shot by each camera; wherein the target photographing instruction is a photographing instruction issued by the user to the preview window of the specified camera when the user needs to determine whether a patient's wound is displayed in the sub-area; Displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image; wherein the marking points are used to locate the wound; In response to a user's position adjustment operation on a target marking point among the plurality of marking points, adjusting the position of the target marking point; In response to receiving the size information calculation instruction, based on the images captured by each camera and the current image coordinates of each target marking point, convert the current image coordinates of each target marking point into target coordinates in the world three-dimensional coordinate system; Based on the target coordinates of each target marking point, the size information of the wound is calculated, and the size information is associated with the target image.

2. The method according to claim 1, characterized in that The method of packaging each camera of the binocular camera includes: Establish a mapping relationship between the physical ID of each camera and the target ID; wherein the target ID is the ID of the target camera; The step of controlling each camera of the binocular camera to be turned on simultaneously in response to the turn-on instruction for the target camera includes: In response to an on instruction for a target camera, the physical ID to which the target ID of the target camera is mapped is determined from the mapping relationship, and each camera with the determined physical ID is controlled to be turned on simultaneously.

3. The method according to claim 1, characterized in that The setting method of the highlighting effect of the sub-area of ​​the preview window of the specified camera includes: A mask is set for an area outside the sub-area of ​​the preview window of the specified camera to highlight the sub-area.

4. The method according to claim 1, characterized in that The shape of the sub-region is a specified graphic shape; The determination method of the multiple initial positions includes: A plurality of initial positions are determined according to a position determination method that matches the shape category of the specified graphic shape.

5. The method according to claim 4, characterized in that The specified graphic shape is a rectangle; Accordingly, the multiple initial positions are determined according to the position determination method that matches the shape category of the specified graphic shape, including: The midpoint position of each side of the border of the sub-region is determined to obtain multiple initial positions.

6. The method according to any one of claims 1 to 5, characterized in that: The all-in-one device is also provided with a level; In response to the target photographing instruction, each camera is controlled to photograph an image, and before obtaining the image photographed by each camera, the method further includes: The level is started, and in the preview window of the designated camera, guidance information on whether the device is in a horizontal posture, which is identified by the level, is displayed to guide the user to adjust the posture of the all-in-one device to a horizontal posture.

7. The method according to any one of claims 1 to 5, characterized in that: Each marker point belongs to multiple groups, each group contains two marker points, and different groups contain different marker points; In response to a user's position adjustment operation on a target marking point among the plurality of marking points, adjusting the position of the target marking point comprises: In response to a position adjustment operation of a user on a target mark point among the multiple mark points, displaying a display effect of the target mark point according to a target display mode, and adjusting the position of the target mark point according to an adjustment track indicated by the position adjustment operation; The target display method includes enlarging and displaying the target mark point, and the target mark point and another mark point in the group to which the target mark point belongs have a connecting line.

8. The method according to any one of claims 1 to 5, characterized in that: The method of converting the current image coordinates of each target marking point into the target coordinates in the world three-dimensional coordinate system based on the images captured by each camera and the current image coordinates of each target marking point includes: Determine the marking points corresponding to the target marking points in the image captured by another camera other than the designated camera, and obtain the designated marking points; Based on the current image coordinates of each target marking point, the current image coordinates of each designated marking point and the reference file, the current image coordinates of each target marking point are converted into target coordinates in the world three-dimensional coordinate system; wherein the reference file is a file used to convert image coordinates into coordinates in the world three-dimensional coordinate system.

9. The method according to any one of claims 1 to 5, characterized in that: The all-in-one device also includes a QR code scanning unit; The method further comprises: Scanning the patient's QR code by a QR code scanning unit, and obtaining the patient's information based on the QR code; The patient information is bound to a target image associated with the size information.

10. An image processing device, characterized in that: A main controller applied to an all-in-one device, wherein the all-in-one device is also provided with a binocular camera and a display unit; the device comprises: A first control module, configured to control the cameras of the binocular camera to be turned on simultaneously in response to a start instruction for a target camera, and to display preview windows of the cameras of the binocular camera through the display unit in a predetermined display manner; wherein the target camera is a logical camera obtained by packaging the cameras of the binocular camera; the predetermined display manner is to make the preview window of the designated camera of the binocular camera visible to the user, and the preview window of another camera other than the designated camera is invisible to the user, and there is a sub-area with a highlighted display effect in the preview window of the designated camera, and the acquisition range corresponding to the sub-area belongs to the overlapping field of view range of each camera; A second control module is used to control each camera to capture images in response to a target capture instruction to obtain images captured by each camera; wherein the target capture instruction is a capture instruction issued by the user to the preview window of the specified camera when the user needs to determine whether a patient's wound is displayed in the sub-area; A setting module, used for displaying the target image captured by the designated camera in the display unit and setting marking points at multiple initial positions in the target image; wherein the marking points are used for locating the wound; A position adjustment module, configured to adjust the position of a target marking point among the plurality of marking points in response to a user's position adjustment operation on the target marking point; A conversion module, configured to respond to receiving the size information calculation instruction and convert the current image coordinates of each target marking point into target coordinates in a world three-dimensional coordinate system based on the images captured by each camera and the current image coordinates of each target marking point; A calculation module is used to calculate the size information of the wound based on the target coordinates of each target marking point, and associate the size information with the target image.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images