A system of food safety management inspection and operating method of the system

Abstract

The present invention provides a food safety management inspection system and a method of operation of the system. For example, a method of operation of a data processing server included in a food safety management inspection system according to the present invention comprises: a reference image information acquisition step of acquiring reference image information for a specific test paper from a mobile terminal; a reference color information extraction step of detecting a first region of interest among the reference image information and extracting a plurality of level-specific reference color information for the specific test paper from the detected first region of interest; a result image information acquisition step of acquiring result image information for the specific test paper from the mobile terminal; a result color information extraction step of detecting a second region of interest among the result image information and extracting result color information for the specific test paper from the detected second region of interest; a step of selecting a comparison color model to be used for color comparison among a plurality of color models using at least one of the reference color information and the result color information; and a step of extracting specific level information determined to have the highest similarity to the result color information among a plurality of levels for the specific test paper based on a predetermined weight for each of the three channels of the selected comparison color model. and using the extracted specific level information, it includes the step of generating result data for the specific test paper and transmitting it to the mobile terminal.

Description

Technology Field

[0001] The present invention relates to a food safety management inspection system and a method of operation of the system. More specifically, the invention relates to a food safety management inspection system and a method of operation of the system that can easily measure the inspection results of various test papers used throughout the entire process of food processing and cooking using a mobile terminal and build a database thereof. Background Technology

[0002] A food safety management system encompasses overall technologies for detecting or eliminating harmful food elements or tracking and managing them throughout the entire process of food distribution, processing, and cooking. The market for such food safety management or inspection is estimated to be growing at an average annual rate of approximately 7 to 10% not only domestically but also globally, and it is analyzed that interest in food safety has increased significantly, particularly following the spread of COVID-19.

[0003] With this shift in interest, there is a growing need among not only customers and guests but also business owners and managers for systems to monitor or manage the hygiene conditions of restaurants and food establishments. Prior art literature

[0004] Republic of Korea Published Patent Application No. 10-2022-0085038 (June 21, 2022) The problem to be solved

[0005] The objective of the present invention is to solve the aforementioned conventional problems by providing a food safety management inspection system that compares and analyzes hygiene status information regarding the entire process of food processing and cooking, converts the results into data, and stores the results, and a method of operation of the system. means of solving the problem

[65535] The present invention relates to a method of operation of a data processing server included in a food safety management inspection system, wherein the method of operation of the server comprises: a reference image information acquisition step of acquiring reference image information for a specific test paper from a mobile terminal; a reference color information extraction step of detecting a first region of interest among the reference image information and extracting a plurality of level-specific reference color information for the specific test paper from the detected first region of interest; a result image information acquisition step of acquiring result image information for the specific test paper from the mobile terminal; a result color information extraction step of detecting a second region of interest among the result image information and extracting result color information for the specific test paper from the detected second region of interest; a step of selecting a comparison color model to be used for color comparison among a plurality of color models using at least one of the reference color information and the result color information; and a step of extracting specific level information determined to have the highest similarity to the result color information among a plurality of levels for the specific test paper based on weights predetermined for each of the three channels of the selected comparison color model using a first mathematical formula. and may include the step of generating result data for the specific test paper using the extracted specific level information and transmitting it to the mobile terminal.The data processing server can detect the first region of interest by normalizing a reference image obtained from the mobile terminal, converting the normalized image into a grayscale image, converting the grayscale image into a binary image, extracting an outline from the binary image information, extracting straight lines from the extracted outlines, determining four points where the extracted straight lines intersect as the points of a rectangle, storing the coordinates of the rectangle in a vector, extracting the widest rectangular area based on this, and performing distortion correction on the widest rectangular area. The step of selecting the comparison color model may include: a step of calculating the standard deviation per channel in each color model for at least one of the reference color information and the result color information; and a step of selecting the color model with the largest sum of standard deviations per channel among the plurality of color models as the comparison color model. The plurality of color models may include an RGB (Red Green Blue) color model composed of a Red channel, a Green channel, and a Blue channel; It may include an HSI color model composed of a Hue channel, a Saturation channel, and an Intensity channel; and a Lab color model composed of a Lightness channel, a channel defined based on Green-Magenta, and a channel defined based on Blue-Yellow. The first mathematical formula is. However, the above w1, w2, and w3 are pre-set weight values ​​for each color model, and for the RGB color model, w1 = w2 = w3 is set, and for the HIS color model and Lab color model, w1 > w2 = w3 is set, and stick CH1 / CH2 / CH3 is a color value separated by each channel by converting result color information into a selected comparison color model, and target CH1 / CH2 / CH3 may be a color separated by each channel by converting multiple level-specific reference color information for a specific test paper into a selected comparison color model. The method may further include an identification information acquisition step of acquiring information identifying the specific test paper from the mobile terminal. The identification information acquisition step may be performed before the reference image information acquisition step or simultaneously with the reference image information acquisition step. The specific test paper may include a residual chlorine test paper that displays a color corresponding to the residual chlorine level of the surface of an object in contact with the specific test paper. The method may further include a step of updating a database using result data for the specific test paper. Effects of the invention

[0014] According to the present invention, a user can easily check the results of test papers used throughout the entire process of food processing and cooking using a mobile terminal, and can store them in a database. Brief explanation of the drawing

[0015] FIG. 1 is a diagram illustrating the configuration of a food safety management inspection system according to one embodiment of the present invention. FIG. 2 is a diagram showing the operation method of a food safety management inspection system according to one embodiment of the present invention. FIG. 3 is a diagram showing an example of levels for a specific test paper applicable to the present invention. FIG. 4 is a flowchart illustrating a method for detecting a region of interest according to an example of the present invention. Figure 5 is a diagram showing the result value detected as a region of interest according to the present invention. FIG. 6 is a flowchart briefly illustrating a color comparison method according to an example of the present invention. Specific details for implementing the invention

[0016] Hereinafter, an embodiment of a food safety management inspection system and a method of operation of the system according to the present invention will be described with reference to the attached drawings. At this time, the present invention is not limited or restricted by the embodiments. Furthermore, in describing the present invention, specific descriptions of known functions or configurations may be omitted to clarify the gist of the present invention.

[0017] FIG. 1 is a diagram illustrating the configuration of a food safety management inspection system according to one embodiment of the present invention.

[0018] As illustrated in FIG. 1, the food safety management inspection system according to the present invention may include a mobile terminal (10), a data processing server (20), and a DB (database, 30).

[0019] In the present invention, the mobile terminal (10) may be implemented as a portable terminal such as a computer, smartphone, or tablet PC (Personal Computer) capable of connecting to a server or other terminal via a network. At this time, the computer may include a desktop, laptop, etc., on which a web browser or related application (e.g., application, etc.) is installed, and the portable terminal may include a wireless communication device capable of wireless connection via a communication system such as 4G LTE (Long Term Evolution), 5G, or Wi-Fi. Additionally, in the present invention, the network may be implemented as a wired network such as a Local Area Network (LAN) or a Wide Area Network (WAN), or a wireless network such as a mobile communication network or a satellite communication network. Accordingly, the mobile terminal (10) can transmit and receive signals to and from a data processing server (20) via a communication system supported by the network.

[0020] In the present invention, the data processing server (20) can build and operate a web page or application service that interacts with a mobile terminal (10). More specifically, the data processing server (20) can provide various functions, such as database data using various test papers used in the entire process of food processing and cooking by date or time period through the web page or application it operates, and provide the information to the user.

[0021] FIG. 2 is a simplified diagram showing the operation method of a food safety management inspection system according to one embodiment of the present invention.

[0022] Before the detailed description below, the definitions of the technical expressions described in the present invention will be explained in detail.

[0023] In the present invention, the term 'image information' may include images, video, frame images, picture information, photographic information, etc.

[0024] In the present invention, the term 'test paper' may include test papers capable of measuring pH (potential of hydrogen or power of hydrogen, hydrogen ion concentration index), residual chlorine, temperature, etc., as shown in the table below.

[0025] [Table 1]

[0026]

[0027] In the present invention, the term 'reference image information' for a specific test paper may refer to image information regarding a reference color chart separately printed for the specific test paper. In other words, the 'reference image information' may refer to image information representing result values ​​by color for the specific test paper. Generally, the 'reference image information' may be displayed on the instruction manual, packaging, packaging box, etc., of the specific test paper. Accordingly, a mobile terminal (10) according to the present invention may capture the displayed image through various means, such as a camera application, and provide it to a data processing server (20).

[0028] In the present invention, the term 'multiple levels' for a specific test paper may refer to reference values ​​(or identifiers) that divide the result values ​​of the specific test paper into multiple units.

[0029] FIG. 3 is a diagram showing an example of levels for a specific test paper applicable to the present invention.

[0030] If the specific test paper described above is a test paper for measuring residual chlorine concentration (or pH concentration), the specific test paper may have multiple levels classified from pH 1 to 14 as shown in FIG. 3. At this time, reference color information corresponding to each level (or pH value) may be set as shown in FIG. 3.

[0031] In the present invention, "receiving in real time" may refer to receiving data at the same time as it occurs. In one embodiment, receiving an image in real time may refer to receiving the image as soon as it is captured through an image sensor. Likewise, "providing in real time" may refer to providing data at the same time as it occurs.

[0032] Returning to FIG. 2, the operation method of a food safety management inspection system according to one embodiment of the present invention is described. A mobile terminal (10) according to the present invention can transmit reference image information for a specific test paper and result image information for the specific test paper to a data processing server (20). In response, the data processing server (20) can obtain the reference image information from the mobile terminal (10) (S210) and obtain the result image information (S220).

[0033] In the present invention, each of the above-described steps S210 and / or S220 may be performed in real time. More specifically, the data processing server (20) may receive reference image information for the above-described specific test paper and / or result image information for the above-described specific test paper from the mobile terminal (10) in real time through a dedicated webpage or application. In other words, the data processing server (20) may control (or restrict, force) the mobile terminal (10) to provide only image information captured in real time to the data processing server (20), rather than image information captured in advance, and accordingly, the mobile terminal (10) may transmit the reference image information and / or result image information to the data processing server (20) in real time.

[0034] In another embodiment applicable to the present invention, steps S210 and S220 are illustrated separately in FIG. 2, but the steps may be performed simultaneously. For example, a mobile terminal (10) may transmit the reference image information and the result image information described above to a data processing server (20) simultaneously after capturing the reference image information and the result image information in advance. At this time, the data processing server (20) may determine that the image information is valid only when the time interval is less than a certain amount (e.g., 5 minutes or less) using the shooting time information of each of the reference image information and the result image information. Based on the above determination result, the data processing server (20) may perform operations according to the food safety management inspection system using the valid image information, or request the mobile device (10) to re-transmit the reference image information and the result image information for a specific test paper.

[0035] In another embodiment applicable to the present invention, the steps S210 and S220 described above may be performed through the transmission and reception of a single image file between a mobile terminal (10) and a data processing server (20). For example, the mobile terminal (10) may capture reference image information for a specific test paper and result image information for the specific test paper as a single photographic image and transmit the photographic image to the data processing server (20).

[0036] The data processing server (20) can derive results based on region of interest detection and color comparison for the image information obtained through step S210 and / or step S220 (S230).

[0037] More specifically, the data processing server (20) according to the present invention detects a first region of interest among the reference image information obtained through step S210, and can extract a plurality of level-specific standard color information for a specific test paper from the detected first region of interest. For example, the data processing server (20) according to the present invention can extract standard color information for each level (or step, numerical value) as shown in FIG. 3.

[0038] Similarly, the data processing server (20) according to the present invention can detect a second region of interest among the result image information obtained through step S220, and extract result color information for a specific test paper from the detected second region of interest.

[0039] In the above-described operation, the data processing server (20) may utilize various methods for detecting regions of interest. For example, the data processing server (20) according to the present invention may detect a region of interest from the corresponding image information through a region of interest detection procedure such as FIG. 4.

[0040] FIG. 4 is a flowchart illustrating a method for detecting a region of interest according to an example of the present invention, and FIG. 5 is a diagram illustrating a result value detected as a region of interest according to the present invention.

[0041] As illustrated in FIG. 4, a data processing server (20) can obtain a camera image (e.g., reference image information, result image information, etc.) from a mobile terminal (10). Then, the data processing server (20) can normalize the image (e.g., normalize to 512 * 512) and convert the normalized image into a grayscale image. Then, the data processing server (20) can convert the grayscale image into a binary image. Then, depending on the embodiment, noise removal (e.g., morphology) may or may not be applied to the binary image.

[0042] Next, as shown in FIG. 5(a), the data processing server (20) can extract contours from the image information obtained by the method described above and extract intersecting straight lines therefrom. A Hough Transform method may be applied for this purpose. Next, the data processing server (20) can determine four points where the extracted straight lines intersect as each point of a rectangle.

[0043] The data processing server (20) can store the coordinates of the rectangle obtained through the method described above in a vector, extract the widest rectangular area based thereon, and perform distortion correction on the rectangle according to the embodiment. Through this, the data processing server (20) can detect a region of interest among the entire image information as shown in FIG. 5(b).

[0044] Through the various methods described above, the data processing server (20) can extract multiple level-specific standard color information for a specific test paper (e.g., FIG. 5), and separately extract result color information for the specific test paper. Subsequently, the data processing server (20) can derive a result for the specific test paper by comparing the standard color information and the result color information.

[0045] As an example applicable to the present invention, a data processing server (20) may select a comparison color model to be used for color comparison among a plurality of preset color models by using at least one of the above-described reference color information and result color information.

[0046] According to a preferred embodiment applicable to the present invention, the above-described plurality of color models may be pre-set as the following color models:

[0047] (1) RGB (Red Green Blue) color model

[0048] It is a color model consisting of a red, green, and blue channel, which expresses color using the three primary colors of light.

[0049] (2) HSI color model

[0050] It is a color model composed of a Hue, Saturation, and Intensity channel, which represents color using the three properties of light.

[0051] (3) Lab color model

[0052] It is a color model composed of a Lightness channel, an a channel defined based on Green and Magenta, and a b channel defined based on Blue and Yellow, based on the antagonism theory of human vision.

[0053] In the present invention, the method for selecting the above-described comparison color model may include the following two steps:

[0054] 1) A step of calculating the channel-specific standard deviation in each color model for at least one of the reference color information and the result color information.

[0055] 2) A step of selecting the color model with the largest sum of channel standard deviations among multiple color models as the comparison color model.

[0056] FIG. 6 is a flowchart briefly illustrating a color comparison method according to an example of the present invention.

[0057] As illustrated in FIG. 6, the data processing server (20) according to the present invention can extract reference color information from reference image information. In addition, the data processing server (20) can extract result color information from result image information.

[0058] Next, the data processing server (20) can perform noise removal and / or shadow area correction on the reference color information and result color information extracted through the method described above, and convert each image information into the three color models described above. The data processing server (20) can separate each channel component in each color model and extract the color value of each channel. Next, the data processing server (20) can calculate the standard deviation per channel in each color model and select the color model with the largest standard deviation as the comparison color model. More specifically, as described above, the data processing server (20) according to the present invention can calculate the standard deviation per channel in each color model for at least one of the reference color information and result color information, and select the color model with the largest sum of standard deviations per channel among a plurality of color models as the comparison color model.

[0059] Next, the data processing server (20) can compare reference color information and result color information based on a comparison color model and extract specific level information that is determined to have the highest similarity to the result color information among multiple levels for a specific test paper.

[0060] As a specific example, the data processing server (20) can use the following mathematical formula to extract specific level information that is determined to have the highest similarity to the result color information among multiple levels for a specific test paper, based on predetermined weights for each of the three channels of the selected comparison color model. In the following mathematical formula, w1, w2, and w3 represent pre-set weight values ​​for each color model. For example, for the RGB color model, w1 = w2 = w3 may be set, and for the HIS color model and Lab color model, w1 > w2 = w3 may be set. Additionally, stick CH1 / CH2 / CH3 represents color values ​​separated by each channel by converting the result color information into the selected comparison color model, and target CH1 / CH2 / CH3 represents color values ​​separated by each channel by converting the multiple level-specific standard color information for a specific test paper into the selected comparison color model.

[0061] [Mathematical Formula]

[0062]

[0063] FIG. 7 is a diagram showing an example of the results of color data analysis and similarity calculation according to the present invention.

[0064] As illustrated in FIG. 7, the data processing server (20) can compare the acquired result color information with reference color information for at least three targets (or at least three levels) and extract level information corresponding to the color with the highest similarity as a result value.

[0065] When result data is obtained through the method described above, the data processing server (20) can transmit the result data to a mobile terminal (10) (S240). Additionally, separately from or simultaneously with step S240, the data processing server (20) can transmit the result data to a separate DB (30) (S250).

[0066] Specifically, the result data transmitted by the data processing server (20) in step S240 may include at least one of (1) image information regarding the detected result color information, (2) specific level information determined to have the highest similarity, (3) image information regarding the color information corresponding to the specific level information, and (4) additional information according to the specific level information (e.g., if the pH is 8 or higher, residual detergent remains and additional washing is required; if the pH is 7, the washing was done well, etc.).

[0067] On the other hand, the result data transmitted by the data processing server (20) in step S250 may differ in content from the result data of step S240. Specifically, the result data transmitted by the data processing server (20) in step S250 may include at least one of (1) image information regarding the detected result color information, (2) specific level information determined to have the highest similarity, (3) image information regarding the color information corresponding to the specific level information, (4) reference image information and / or time information in which the result image information was acquired. Through the result data, the data processing server (20) can update the database.

[0068] In an additional embodiment, the mobile terminal (10) may transmit information (also known as identification information) identifying the specific test paper described above to the data processing server (20). In response, the data processing server (20) may obtain information identifying the specific test paper from the mobile terminal (10). For example, the identification information may be information identifying one of the test papers described in the table described above.

[0069] In an embodiment applicable to the present invention, the data processing server (20) may obtain information identifying a specific test paper from the mobile terminal (10) prior to step S210. At this time, the data processing server (20) may obtain image information provided by the mobile terminal (10) (e.g., a QR code capable of identifying a specific test paper), text information entered through a separate input means, selection information regarding one of a plurality of options through a separate input means, etc., and thereby identify the corresponding specific test paper.

[0070] Alternatively, in another embodiment applicable to the present invention, the data processing server (20) may obtain information identifying a specific test paper from the mobile terminal simultaneously with step S210. For example, the mobile terminal (10) may transmit to the data processing server (20) image information (e.g., QR code, etc.) that can identify the specific test paper within the reference image information. Alternatively, while transmitting the reference image information to the data processing server (20), the mobile terminal (10) may also provide separate input information (e.g., selection information for one of a plurality of options through a separate input means, etc.) to the data processing server (20). Accordingly, the data processing server (20) may identify the corresponding specific test paper.

[0071] As described above, a specific test paper applicable to the present invention may include a residual chlorine test paper that displays a color corresponding to the residual chlorine level of the surface of an object in contact with the specific test paper.

[0072] In an additional embodiment, a specific test paper according to the present invention may include a temperature test paper that indicates whether the temperature of the surface of an object in contact with the specific test paper has changed above a certain temperature using a color different from the existing color. According to an embodiment, the certain temperature value may have values ​​such as 65°C, 71°C, 77°C, 82°C, 110°C, 121°C, etc. At this time, a method different from the reference image information and color comparison method for the temperature test paper may be applied. For example, the reference image information for the temperature test paper may correspond to default temperature test color information, and accordingly, a color comparison method may be applied to compare whether the result color information differs from the default temperature test color information by more than a certain threshold value. At this time, if the result color information is determined to be different from the default temperature test color information, it means that the surface of the object in contact with the temperature test paper has changed to a temperature above a certain level, which may mean that the object has been properly cleaned.

[0073] According to the above-described embodiment, the data processing server (20) uses identification information received from the mobile terminal (10) to extract specific level information that is judged to have the highest similarity through color comparison according to the above-described embodiment, such as FIG. 2, if the specific test paper is a residual chlorine test paper, unlike the above-described embodiment, it can extract whether the cleaning was done properly based on whether the result color information differs from the basic temperature test color information by more than a certain threshold value. The data processing server (20) can generate the extracted information as result data and provide it to the mobile terminal (10) and / or DB (30).

[0074] As described above, preferred embodiments of the present invention have been explained with reference to the drawings; however, a person skilled in the art may make various modifications or changes to the present invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0075] 10: Mobile device 20: Data processing server 30: DB

Claims

Claim 1 A method of operation of a data processing server included in a food safety management inspection system comprises: a reference image information acquisition step of acquiring reference image information for a specific test paper from a mobile terminal; a reference color information extraction step of detecting a first region of interest among the reference image information and extracting a plurality of level-specific reference color information for the specific test paper from the detected first region of interest; a result image information acquisition step of acquiring result image information for the specific test paper from the mobile terminal; a result color information extraction step of detecting a second region of interest among the result image information and extracting result color information for the specific test paper from the detected second region of interest; a step of selecting a comparison color model to be used for color comparison among a plurality of color models using at least one of the reference color information and the result color information; and a step of extracting specific level information that is judged to have the highest similarity to the result color information among a plurality of levels for the specific test paper based on weights predetermined for each of the three channels of the selected comparison color model using a first mathematical formula.The method includes the step of generating result data for the specific test paper using the extracted specific level information and transmitting it to the mobile terminal, wherein the data processing server normalizes the reference image obtained from the mobile terminal, converts the normalized image into a grayscale image, converts the grayscale image into a binary image, extracts an outline from the binary image information, extracts straight lines from the extracted outline, determines four points where the extracted straight lines intersect as points of a rectangle, stores the coordinates of the rectangle in a vector, extracts the widest rectangular area based thereon, and performs distortion correction on the widest rectangular area to detect the first region of interest; and the step of selecting the comparison color model includes the step of calculating the standard deviation per channel in each color model for at least one of the reference color information and the result color information. The method includes the step of selecting the color model with the largest sum of standard deviations per channel among the plurality of color models as the comparison color model, wherein the plurality of color models include: an RGB (Red Green Blue) color model composed of a Red channel, a Green channel, and a Blue channel; an HSI color model composed of a Hue channel, a Saturation channel, and an Intensity channel; and a Lab color model composed of a Lightness channel, an a channel defined based on Green - Magenta, and a b channel defined based on Blue - Yellow, and the first mathematical formula is; A method of operation of a data processing server, wherein w1, w2, and w3 are pre-set weight values ​​for each color model, and for the RGB color model, w1 = w2 = w3 is set, and for the HIS color model and Lab color model, w1 > w2 = w3 is set, stick CH1 / CH2 / CH3 is a color value separated by each channel by converting result color information into a selected comparison color model, and target CH1 / CH2 / CH3 is a color value separated by each channel by converting multiple level-specific reference color information for a specific test paper into a selected comparison color model. Claim 2 A method of operation of a data processing server, characterized in that, in claim 1, it further includes a step of obtaining identification information, wherein information identifying the specific test paper is obtained from the mobile terminal. Claim 3 A method of operation of a data processing server, characterized in that, in claim 2, the identification information acquisition step is performed before the reference image information acquisition step or simultaneously with the reference image information acquisition step. Claim 4 A method of operation of a data processing server according to claim 1, wherein the specific test paper comprises a residual chlorine test paper that displays a color corresponding to the residual chlorine level of the surface of an object in contact with the specific test paper. Claim 5 delete Claim 6 delete Claim 7 A method of operation of a data processing server, characterized in that, in claim 1, it further includes the step of updating a database using result data for the specific test paper. Claim 8 delete

Images